localfirst.fm

A podcast about local-first software development

Listen

Conference

#14 – Matthew Weidner: Architectures for Central Server Collaboration

The guest of this episode is Matthew Weidner, a computer science PhD student at Carnegie Mellon University focussing on distributed systems and local-first software. Matthew has recently published an extensive blog post about architectures for central server collaboration which is explored in depth in this conversation comparing different approaches such as CRDTs and event sourcing.

Mentioned in podcast

Links:

Thank you to Expo and Rocicorp for supporting the podcast.

Transcript

#14 – Matthew Weidner: Architectures for Central Server Collaboration
00:00And this also feeds into features that you might want to give to your
00:02users, especially in productivity apps.
00:04You want to have that change history where you can see what was everyone doing.
00:07You also want to have undo's.
00:09basically what you can do for undo is when you create this action or this mutation
00:13describing, the high level intent, you can also tag along with it, a mutation saying,
00:17here's how to undo this operation later.
00:19And then you store that somewhere and then you just have a queue somewhere in
00:22your app that's like the action queue.
00:24You can go through that and undo things in a nice way.
00:27Hopefully, users will be happier with this than if you just, you
00:31know, revert states exactly, ignoring collaborators updates.
00:35Welcome to the local-first FM podcast.
00:37I'm your host, Johannes Schickling, and I'm a web developer, a
00:40startup founder, and love the craft of software engineering.
00:43For the past few years, I've been on a journey to build a modern, high quality
00:47music app using web technologies.
00:49And in doing so, I've been falling down the rabbit hole of local-first software.
00:54This podcast is your invitation to join me on that journey.
00:57In this episode, I'm speaking to Matthew Weidner, a computer science
01:01PhD student at Carnegie Mellon University, focusing on distributed
01:05systems and local-first software.
01:07Matthew has recently published an extensive blog post about architectures
01:12for central server collaboration, which we explore in depth in this conversation,
01:17comparing different approaches, such as CRDTs and event sourcing.
01:21Before getting started, also a big thank you to Rocicorp and
01:24Expo for supporting this podcast.
01:27And now my interview with Matthew.
01:29Hey, Matthew.
01:30Thank you so much for coming to the show.
01:32How are you doing?
01:33I'm good.
01:34Yeah.
01:34Thanks for inviting me.
01:36Yeah.
01:36Super excited to, to have you here.
01:38I think, our shared friend, Geoffrey Litt introduced us and he and, Matt Wondlaw
01:44and a few others have, when you were writing this blog post, the architectures
01:48for central collaboration, all of my friends shared this blog post with me.
01:53And it has since, like, served as a really, really reliable and
01:57good foundation to just provide an orientation around, yeah, how do syncing
02:03systems, et cetera, how do they work?
02:06So this has been the, the initial touch point for me, but would you
02:10mind briefly introducing yourself?
02:12sure.
02:13Yeah.
02:13So I'm Matthew.
02:14I'm a researcher and developer.
02:16I've been thinking about, local-first software, more generally the problem
02:19of how do we make collaborative software easier to program.
02:23So that's been, I guess, five years of PhD work and now working full time on a
02:28collaborative app, at a small company.
02:30And yeah, the, the question for me has always been, how can we make
02:33building a collaborative app in the style of Google Docs or Figma
02:36as easy as making a smartphone app or a local only desktop app?
02:41Amazing.
02:42I'm curious, what led you, like when you say five years ago, you started working
02:46on this, what led you to, to that point?
02:48What motivated you to, to look into this?
02:51yeah, so it actually started a little earlier.
02:53So six years ago, I was doing a master's degree at the University of Cambridge.
02:57I had to pick a master's thesis project, and some of the Ph.
02:59D.
03:00students talked about what their lab group was doing, the TrueData group,
03:03where they were working on an end to end encrypted version of Google Docs.
03:06The idea is that some professions, like lawyers or journalists, they want the
03:09collaboration of Google Docs, but they don't trust their data to a third party.
03:13where the, you know, the employees can look at it or
03:14it's on someone else's servers.
03:16So they wanted this end to end encryption where you say only
03:18you and your collaborators can read the unencrypted data.
03:22So I thought this sounded like a really interesting project.
03:23I just joined them for my master's thesis.
03:25Turned out to be working with Alastair Beresford and Martin Kleppmann, um,
03:30mostly on the cryptography side.
03:31Then after that, I decided that actually the collaboration side
03:34sounded more interesting, and I wanted to work on that for my PhD.
03:37Very interesting.
03:38What did the technology landscape at that point look like?
03:41I mean, today there's like Automerge and quite a few other technologies
03:45that already try to attempt this.
03:47what did the technology landscape back then look like?
03:50So this was before the local-first essay.
03:52I think I actually saw a draft of the local-first essay that
03:56year, now as a master's student.
03:57Automerge I believe had started, YJS had started, but I hadn't
04:01heard of people using it yet.
04:03but yes, people were just getting started to use this idea of.
04:07collaborative data structures for the web, not necessarily with central
04:10servers like these CRDT libraries were just getting started, and I don't
04:15know if the local-first world had really even started yet at that point.
04:19Right.
04:19Yeah.
04:20I think there are so many people who thought about similar problems
04:23over like decades before then.
04:25There was like CouchDB and PouchDB and like a lot of great minds
04:29already thought about this, but I feel like the real momentum
04:32started with the local-first essay.
04:35So I'm curious, take me through a little bit of like the, the
04:37five years working on that.
04:40What were some of the milestones?
04:42How did you go about starting this in the first place?
04:45Sure.
04:45So the, the main things I was coming at it from a more academic perspective, like
04:49I really have a theory math background.
04:51So I was looking at the, the theory of CRDTs, these conflict
04:54free replicated data types.
04:56Which, sort of, the idea is that it's a data structure that's
04:59copied on multiple devices.
05:01You put your data in it, like your app's data, and then one user can change their
05:04copy of the data whenever they want.
05:06At some point later, you'll sync up in the background and come to
05:09a convergent copy where everyone's looking at the same document again.
05:12This is really designed for the sort of peer to peer model where you don't
05:15necessarily have central authority, it's just everyone updating their own data.
05:19and also this local-first spirit, where you always update the local copy of
05:21your data first, and then you talk to everyone else and say, here's my changes.
05:25So I spent the first year really just reading the papers in that field.
05:29So there's a classic paper by Mark Shapiro right now for 2011, a lot of papers by
05:34Carlos Vaccaro and his collaborators, yeah, just trying to learn what are these
05:38data structures, what can we do with them.
05:41Got it.
05:41And so after that, you started your own implementations of CRDTs.
05:46And was there any sort of reference app that you oriented this around?
05:50Not really.
05:51So there's actually, there's a reference CRDT.
05:53So we started with this paper, which is very theoretical about this way
05:56that you could maybe combine two CRDTs.
05:59So the example we use, which is a bit silly, is if you have.
06:02a number that you can add things to, like maybe a bank account balance
06:06you can add to, you can also multiply to if you're applying the interest.
06:09How do you combine these two operations in a single CRDT that can
06:12be updated with either add or multiply?
06:15So then my advisor had this idea, let's implement this in a library.
06:18and there that already set some sort of unique design principles,
06:23which is that we're going to assume you're making your own CRDTs.
06:26It's not just a collection of CRDTs we give to you, like map, list, et cetera,
06:30actually going to be whatever, and then some way to combine them together.
06:34So that was really the starting point, is that we want to make a place where you
06:38can make your own CRDTs and compose them.
06:40I don't think we really had a specific application in mind at the beginning.
06:44Was that technology ever released or open source or talked about in some way?
06:50So we did make a open source library about it.
06:52It's called Collabs.
06:53So it's written in TypeScript.
06:55we have a documentation site.
06:56I think it's collabs.readthedocs.Io.
06:59it's definitely still an academic project.
07:01So it's really about, here are these data structures that you can play
07:04with and you can make your own things.
07:06we do have some basic demo apps, like your basic, uh, You know, text editor.
07:11there's a to do list sort of thing somewhere.
07:13And then there is an archive paper about it that you can read, which goes into
07:16more detail about the system design and why we did things the way we did.
07:20Got it.
07:20And so it sounds like you've really gone super deep on this, mostly
07:25oriented from the CRDT side of things.
07:28But, as you read the papers, as you were working on this, you also got
07:33a better understanding of the larger space and the other approaches.
07:37And I think you got more curious about the other approaches and this
07:40is what you've laid out so clearly and brilliantly in this blog post that will
07:45be linked in the, in the show notes.
07:47And I highly recommend anyone who's listening to read it in depth, if
07:51you're curious about those topics.
07:53so the, the blog post called Architectures for Central Server Collaboration, and
07:58it provides a really nice way to think about this, like provides of like a.
08:03Hierarchical structure of what are the design decisions?
08:07What are the trade offs?
08:08What are the concerns about the different approaches.
08:11And so I've, I'd love to just go through that step by step.
08:16maybe you want to walk us through it.
08:17Sure.
08:18Let's see.
08:19Yeah.
08:19So the, the idea of this blog post is we're thinking about.
08:24Real time collaborative apps.
08:25So these are apps like Google Docs, Figma, Notion, that sort of thing.
08:29And sort of the distinguishing feature of these apps compared to
08:32more traditional web apps is that, you know, when you make a change, it
08:36updates your local copy immediately.
08:38It's not just click a button, go back to the server, get a
08:41new web page and show it to you.
08:43It's click a button and something updates on your own screen
08:45immediately and eventually it'll tell the server what you did.
08:48So this blog post was trying to think about, in general, with these real
08:51time collaborative apps, like, what are we doing in a semantic sense?
08:54Like, what does it mean to be real time collaborative?
08:57And then, what sort of, you know, the high level of how you can implement
09:00that in the most flexible way possible.
09:03And so you've derived a couple of like really nice ways to, to think about
09:08that, like in terms of dimensions and later on you, you can nicely
09:12summarize it in a nice overview table.
09:15would you mind motivating some of the dimensions that you come up with here?
09:20Sure.
09:21Let's see.
09:21So I guess just for context, my own background is, as I said, thinking
09:24about it from a CRDT perspective.
09:26This is very much the perspective if you have some data structures,
09:30which are usually pretty low level, like maps and lists, and you have
09:33some prescribed operations that you can perform on them, and then it'll
09:37sync it for you under the hood.
09:39And then also in the CRDT model, it's usually not really assuming a central
09:43server, where the central server is doing basically the same thing as the clients.
09:47So what the dimensions are thinking about is, okay, what can we do that's
09:51different from just the CRDT model?
09:53And there is Yeah, there's really three dimensions.
09:56I guess maybe the most interesting one is the is how you describe
10:00operations on the collaborative state.
10:03So you have sort of the, the database or key value store model, which is,
10:06you have these low level state changes.
10:09Like when I check a box in to do list, that's creating a row in a database
10:14that says, you know, to do list checked, true, that sort of thing.
10:18And then there's also this opposite model, which is sort of the more event sourcing
10:21approach where you have these high level operations, sometimes called mutations.
10:26And this is where, when you change the data, you're actually telling the server
10:29exactly what the user's intent was.
10:31You say, the user wants to check this box and make it true, and
10:34then you broadcast that high level intent back to the other users.
10:38And tell them what to do and how to update their state.
10:41And I think this is also like this distinction between the
10:45intent of a mutation and the, the change more directly.
10:50I think this can be, a little bit of a subtle difference for
10:55people who haven't built something with either approaches yet.
10:59But, uh, I think to draw an analogy from the web world, When you're working
11:05with something like Redux, this is where I'm not sure whether you ever
11:09built some, some front end apps with Redux, but this is where you have, for
11:12example, if I remember correctly, the, the concept of an, of an action, which
11:17is basically the idea of an event where you declaratively say like, okay, there
11:22is an action or there is an event for, someone wants to complete this to do.
11:29Then further down the road, there's like a reducer, which then in, for example,
11:34maintains a list of to-dos and maybe kicks it out or maybe, overrides a property
11:41in the to-dos array and says something is done as opposed to the other approach
11:46where you directly mutate the, the state.
11:50Which is, for example, in the web world, we're using something like MobX, etc.
11:55And so now we're talking here about the equivalence for distributed states,
12:00and where CRDTs, I think, give us more the analogy, this might be a stretch,
12:05but give us more of like an equivalent of like something like MobX, where
12:09you mutate the state more directly.
12:11And the CRDT underpinnings nicely make that principled constrain
12:16you in the in the right way and then also distribute the state.
12:20Did I summarize this in the right way?
12:23Yes, good description.
12:25Maybe another way to think about it that's in more illustrative
12:28than to do list is to think about like the the video game example.
12:31So for example in a video game if you press an arrow key on your keyboard you
12:35can do sort of the high level intent is I want my character to move forward.
12:40And then your game server will interpret that intent.
12:42It'll try to move your character forward, but if there's a wall
12:44in the way, it'll stop you.
12:45And if you step on a pressure plate, it'll do something.
12:48and then ultimately compute the actual state changes, which are
12:51the low level things of like, what coordinates are my player at now?
12:55what is the state of the world in terms of, you know,
12:57doors that are open or closed?
12:58And it'll send those low level state changes back to clients.
13:02So that's another example of this distinction between high
13:04level versus low level intent.
13:06Right, and I think this is now also a really important distinction because in
13:11the Redux or MobX example, it's, like, all of that is happening on the local device.
13:18There's no cheating in that regard, but when you're talking about games,
13:22they can actually be cheating.
13:23And how do you prevent that particularly in a multiplayer context?
13:27And this is where you, what do you do on the client and what do you do on the
13:32server, maybe need to be different things where the server acts more than authority.
13:38And the client rather provides, instructions as opposed to providing
13:42the authoritative source of truth for the actual state of a world.
13:47And so this is where the intent is not equal to the reality
13:53that is coming out of it.
13:55And I think this is nicely illustrated in your article through this game example,
14:01where you can basically send to the server, like, Hey, I want to move forward.
14:05The server knows where you were before, and the server tells you
14:09afterwards, like, now you're here.
14:11The client locally can probably, if everything is in an okay state, has
14:16probably already arrived at the same conclusion, but, at least this way the
14:21client can't override to say the player position is somewhere in an illegal state.
14:27Maybe this sort of transitions into the next point or another
14:30dimension in the article.
14:32Which is, what does the server actually do when it receives
14:35an operation, in particular an operation that's out of date?
14:38So the classic example is if you have a like counter, like a post has
14:42some number of likes on it, if it has six likes and I send a command
14:45to the server that says, I like it, change the number of likes to seven.
14:48But what if someone else also liked the post in the meantime, and their
14:52like made it to the server first?
14:54So now the like count's already seven, I don't want to set it to seven
14:56again, I want to increase it to eight.
14:58And there's a, yeah, so basically there's a few philosophies in how the
15:01server should process this operation so that it still makes sense.
15:04I mean, technically it's legal to keep the original operation as
15:08just set the count to 7, but that's not really what the users expect.
15:11So the one philosophy, sort of the CRDT way, is to say, I'm going to phrase
15:15my operations in such a way that the server will know what I want it to
15:19do, And it'll do the correct thing.
15:21So for a light counter, the classic way is you say, increase the light count by one.
15:26The server can get that, and even if the count has gone up since what you
15:29originally thought it was, it's still going to add one and do the proper thing.
15:32So you're going to end up with eight lights instead of seven.
15:34And sort of the other spirit is the operational transformation spirit.
15:38So this is an older technique for collaborative apps that's used by
15:41Google Docs and was developed in the 90s for the Jupyter collaboration system.
15:45And here the spirit is, the server is going to look at your operation, it's
15:48going to look at all the intervening operations that you didn't know about but
15:52the server has received already, and it's going to use those to sort of compute
15:56what your new intent is supposed to be.
15:58So this example, you would tell the server, change the like count
16:01to seven, but the server would see that there was an intervening change
16:04the like count operation already.
16:06It's going to rewrite your operation as change the like count to eight, and
16:09actually apply that to its state and send that operation to the other users.
16:13Got it.
16:14So, and this is basically about the, the convergence aspect And I
16:18suppose where this code is running, this can equally work on the
16:22client as well as on the server.
16:25So this is sort of orthogonal to the, the game example case that we talked
16:30about, which is more about the authority.
16:33Yeah.
16:34Yeah.
16:34So this isn't about how does the server.
16:36interpret operations from, like, a correctness permissions perspective.
16:40It's just how does the server handle operations that are sort of
16:43stale, in the sense that the client originally applied them one state,
16:46but by the time they arrived at the server, the state had updated because
16:49other people were doing things.
16:50Now the server has to figure out what to do.
16:52Yes, this is the server side rebasing.
16:55This is where the server has to rebase your operation, or
16:58the incoming operations, on top of whatever its new state is.
17:02And sort of the analogy is to git rebasing, where you might try to apply
17:05a commit on top of some new commits that weren't there when you first tried it.
17:10Got it.
17:11Okay, so that is one dimension that you've nicely dissected
17:15here in this, in this blog post.
17:17what is the next one?
17:19So the next one is the the optimistic local updates on the client.
17:23So now if we assume there's an central server, everyone's taking
17:26these updates, they're sending these operations to the server, the server
17:29knows what the state's supposed to be.
17:31And what you could say is just the traditional, web app model.
17:34If I submit an operation to the server, it processes it, it sends back, sends me
17:38back the result, and now I get to see it.
17:40So if you think like, um, you know, traditional HTML form, you submit your
17:43operation to the server, it gives you a new page back saying what it is.
17:46But with modern apps, we want to do better than that.
17:48We want to say that when I perform an operation on the client, it's going
17:52to update my own state immediately.
17:54And that's an optimistic update because I'm sort of optimistically
17:57assuming that the server is actually going to receive my update.
18:00It's going to process it in the way I expected.
18:02No one else is going to interfere.
18:04this is just a nice property in terms of making the app feel more responsive.
18:07You want to see your key presses immediately.
18:08You want to see that button get checked immediately.
18:10So the question is then, how do we actually do that?
18:13Or, I guess the first question is even, what is the correct answer?
18:17What does it mean to optimistically update my state?
18:20And I guess, yeah, sort of the conclusion I came to that, you know,
18:23people have come to in computer games as well, is that you want to take
18:27the latest state you've received from the server, plus your own optimistic
18:32local operations on top of that.
18:34And that's always what the correct state is.
18:36And even as you receive or perform new operations, you're
18:38just maintaining that state.
18:40Like from your first dimension, which is about server side rebasing, now it's
18:45a lot of the same ideas, but applied on the client where you need to make
18:50the same trade off decisions again, you might come up with different conclusions
18:56based on the server and based on the client, depending on your use cases.
18:59So that, that is the second dimension.
19:03And, then you're, you talk about the, the form of operations.
19:07So how, a state is changing based on mutations, based on state changes.
19:15Can you go a little bit more into, into detail here?
19:18Sure.
19:18Yes.
19:19This is what we were talking about at the beginning, where when you, you check
19:22a box in a to do list, you want to say, Am I updating a row in a database that
19:25doesn't know anything about to do lists, or am I sending a high level mutation
19:28that says, like, this user wants to check the to do list and, you know,
19:32do that action or maybe do something else if that's not valid anymore.
19:36So here we get to choose which form of operations we want.
19:38We want to send these high or low level from the client to the server.
19:42Then once the server updates its state, does it want to send high or
19:45low level changes back to the clients?
19:48yeah, so the video game example is an interesting one where you actually
19:50make different choices usually.
19:52So usually you'll send the high level operations from clients to the server.
19:55You say, I want to move forward, I want to shoot my crossbow.
19:58And then on the way back from the server to the client, usually it
20:01won't send those actual actions.
20:02It'll just send the results, which are changes to some basic key value store.
20:06But you can also make different choices, like you can say, you
20:10know, Git is an example where it's sort of high level mutations.
20:14You're saying, like, I want to, you know, change this text paragraph in
20:17a specific file, and Git will send those exact operations to every client.
20:21It's not going to interpret them at all on the server and change
20:24them into a low level change.
20:26Whereas if you use something like the Firebase database, that's all low level.
20:30You send low level changes to Google servers.
20:32Where you say, I want to, you know, set this key to this value or I want
20:35to delete this object in the database.
20:38And it's going to send that change back to clients without having any idea what
20:41the keys and values actually represent.
20:43That makes sense.
20:44And so I think this is also nicely drawing a boundary between the more declarative
20:51approaches that you have in mutations that you can reason more clearly about,
20:56like in the context of your domain.
20:58But it also only makes sense in the context of your domain.
21:02Whereas with state changes, this is the appeal of CRDTs.
21:06This is you just mutate a document and, the, the underlying mechanics, make
21:12sure that the state changes are behaving in, in a useful way since I, I suppose
21:17like listening to the state changes yourself in your app, that's no fun.
21:22So you really want, a system like CRDTs to make sense of that
21:26. So now with those three dimensions and I go through them again, the
21:30server side rebasing, the optimistic updates and the form of operations
21:34like declarative versus state based, now you've combined all of that in a
21:39really nice, classification table where we get a whole bunch of like matrix
21:45cells here with different technologies.
21:48So, Again, highly recommend, actually reading this and looking at the
21:52beautiful table for yourself, but in the different cells, you've
21:56also filled in a couple of existing technologies and see where they slot in.
22:01So would you mind going through the different technologies and maybe
22:05sharing what's interesting about it?
22:07Sure.
22:08So I guess first I can talk about the one cell just near the bottom, right
22:11in the table, if you're looking at it.
22:12Which is the CRDTxCRDT cell.
22:18So this is basically the place where I spent my most time reading
22:21about CRDTs, working on this academic open source library.
22:24And that's where the operations that users send are really these low level state
22:29changes to some sort of magical replicated database, where you update the database,
22:33like normally on your local device, and it promises to do this synchronization in
22:37the background and make sure that everyone converges to the same state immediately
22:40without really caring about what specifically your data or operations are.
22:44So that some prominent examples.
22:45So Firebase Realtime Database, I think of as an example, also
22:49the CRDT ish libraries, like YJS.
22:52also, yeah, Triplit, InstantDB, those are all sort of in this quadrant
22:56or in this cell thing that we're going to replicate low level changes
23:00for you, just like as they are.
23:02another cell on this table, which is sort of near the bottom left, we
23:05mentioned in the computer game example.
23:07In a computer game, you're going to send these high level actions to the
23:10server, which is going to figure out what to do with them, and then communicate
23:14the state changes back to clients.
23:16that's another interesting cell, both because it's sort of old, like, you know,
23:20this is, starts with the Half Life game engine in the 1990s, so people have been
23:23using this technique forever, just not in web apps, it's in computer games.
23:28But more recently, Replicache implements this model as a data sync
23:32layer for web applications, which I know a number of companies are using.
23:36and I found that really inspirational reading about how Replicache works.
23:39I'm glad to have learned about it.
23:41Right.
23:41And I love like how you compare those technologies.
23:44Both technologies.
23:45I love, love like the Half Life game engine spent way too much
23:49time, playing various Half Life game engine games, where it's very, very
23:54intuitive that if you play, press the W key, which moves you forward.
23:59That's like communicating the intent.
24:01To the server, you don't tell the server like, Oh, I'm at these coordinates.
24:04You just give it like a history of like which keys you pressed
24:08and therefore like how you moved.
24:10and it does some validation of like whether all of that is okay.
24:13And it sends you back the location.
24:16And it's the same about Replicache where you send it a few mutations And on the
24:20Replicache server, it interprets all of that and sends back to you the state
24:25using the server side knowledge, which might be different than the client side
24:29implementation, so it's the authority.
24:31So that is very clear and very nicely laid out here, where you send the intent,
24:36you send the declarative mutations, and the server sends you back some state
24:40changes, as opposed to what you before mentioned, with a CRDT times CRDT,
24:46where Both on the client, on the server, you run the same CRDT convergence.
24:52And, uh, so those two, those two cells are very clear.
24:55Yes, exactly.
24:56And then, yeah, so I guess the remaining cells of the table, they
25:00mostly, they either use state changes in both directions or they use high
25:04level mutations in both directions.
25:06So, let's see.
25:07Two interesting ones.
25:08Automerge in ShareDB.
25:10They're both doing a similar idea to the CRDT libraries, like YJS, where they're
25:15sending these low level state changes around and making sure everyone converges
25:18to the same state, but they have a different way of doing this internally.
25:22So with Automerge, what you're actually doing is you're performing these state
25:26based Automerges that a library is basically a JSON CRDT, but the way
25:30it works is more of an, like an event sourcing model, where you have this
25:35total order of CRDT style operations.
25:38All clients are going to make sure that they eventually
25:40confer to the same total order.
25:42So everyone will agree what operation 1, operation 2, operation 3, etc.
25:46The state is the result of applying all of these operations in that fixed order.
25:50And if, you know, people do operations concurrently on their different devices
25:54because the network's not working, then we'll just sort those operations
25:57into some order later, make sure everyone agrees on the same order,
26:00and that's giving you your state.
26:01Interesting.
26:02So given that Yjs and Automerge, which I think are in the web ecosystem, the, the
26:07two most popular CRDT implementations, they actually do differ in this dimension
26:12of like how state changes are implemented.
26:15again, Firebase, as well as Yjs.
26:17following more strictly the CRDT approach and Automerge using server reconciliation.
26:23is there an example that comes to mind where this, in a example app
26:27use case would differ and where you would use Automerge or Yjs,
26:32intentionally because of this?
26:34I think in terms of the, the external.
26:37the API, or what you see as a user of these libraries, it doesn't really differ.
26:41It's more just in terms of the implementation, I guess, in, in this
26:45totally ordered model like Automerge uses, you don't have to worry as much
26:48about getting the math exactly right.
26:50Like, am I sure that these two operations actually do the same thing
26:53if I apply them in different orders, which is this mathematical requirement
26:57that you have to satisfy for CRDTs.
26:59So that makes it a bit easier on the, to like the correctness and
27:04sureness of the implementation.
27:06Whereas with the YJS or CRDT style, if I'm just going to apply my operations
27:10directly, in principle that can be a bit faster because you don't have to
27:14worry about rewinding your total order of operations and then applying a new
27:18thing and walking it forward again.
27:21That said, usually if you're making a collaborative application with CRDTs,
27:24you don't really need to process more than a handful of operations
27:28every second, so it doesn't matter if it takes a little bit longer.
27:31Got it.
27:32Okay.
27:32That, that makes sense.
27:33So in the CRDT approach, wherever I am currently in my state, I can just apply
27:38on top the existing or the new events.
27:41And, with a server side reconciliation approach, this is where depending
27:45on what the new events are, where they sit in terms of the timeline.
27:49I might need to, uh, Wind back, apply them, and that might take a little
27:54bit longer, but possibly also makes the implementation a bit easier.
27:58Yeah, I guess just one note.
27:59So, you've been saying server side reconciliation.
28:01Automerge does not actually require a server.
28:03It's a completely decentralized model.
28:05The name is just sort of by analogy to what you would do
28:07if you would have a server.
28:09You would put all the things in the order that the server receives them.
28:11Automerge instead infers a sort of order in a decentralized way.
28:15That makes sense.
28:16So, we've now mostly talked about the state changes side of it.
28:21And, we talked about how our optimistic, locally, how are
28:26the state changes applied.
28:28But we didn't talk too much about the mutations times mutations quadrant, which
28:32also has couple of, like, Subsections.
28:36So let's dig a little bit into this one.
28:38Yeah, so this, this mutations, mutations quadrant, this is sort of the event
28:42sourcing idea where instead of sending around low level changes, we're going
28:45to send around the actual user actions, both from users to the server and
28:49from the server back to other users.
28:51So an example would be like, if you do a find and replace operation, or maybe
28:56you rename a variable in VS code, the operation that you're going to send
28:59to the server actually says, you know, rename this variable from foo to bar.
29:03As opposed to a bunch of low level edits where you go through and change
29:06the actual characters, F O O to B A R, in every place they happen to exist.
29:10So this quadrant is interesting because it gives you a lot more flexibility
29:15in terms of what You can communicate this really high level intent, like
29:20code refactors or actions in a computer game, and then the server can interpret
29:25that intent in a reasonable way.
29:27You know, applying permissions, maybe you can see that someone else has also been.
29:31you know, added a new reference to that variable.
29:33So it's going to rename that reference as well.
29:36and you can do this a lot more flexibly as opposed to if you just see the low
29:38level intent and have to sort of, or the low level operations and sort of have to
29:42guess what intent that corresponded to.
29:44So there's a few systems along these lines.
29:47So one of them, which I link here, which is not as well known is called Actyx.
29:51It's actually a company in Europe, which does, Like iot, coordination in factories.
29:58So if you have some, you know, robots moving around a factory floor, they're
30:01talking to each other over the local network and they might say things like,
30:05oh, someone needs to go pick up this box and move it from point A to point B.
30:09one of the robots can say, okay, I'm going to go pick up,
30:11pick up this box and move it.
30:13And that way the other robots know not to move it themselves.
30:15And these, these actions or messages, they just get put into a log that
30:19all the devices in the factory see.
30:21And that way they sort of know what's going on, what tasks are
30:24outstanding, that sort of thing.
30:26Right, and I think one very nice benefit of that as well, is that if there's
30:31some real world stuff happening, and whether in a factory a robot has moved,
30:37or you've now like manufactured a new part, or destroyed a certain thing.
30:43Now you have like a real log of those events.
30:46So in case something goes wrong or in case there's an audit, now you have
30:50some hard facts that you can look at.
30:52So it's not just useful for an app and a machine, but it's also useful for human
30:57purposes to understand what has happened.
31:00Exactly.
31:01Yeah.
31:01And this really feeds into the idea of business logic.
31:04You know, in a lot of applications, we have this.
31:07Business logic that we want to do in terms of, you know, what happens
31:10when a user clicks this button.
31:12And it can often be more complicated than you can express
31:15with simple database changes.
31:17And keeping these actions around gets you really first look at what the, the
31:20business logic was supposed to do and also have the server customize its response.
31:25Like you can check permissions at a very fine grained level.
31:28You can make decisions about, you know, bank balances going below
31:31zero and that sort of thing.
31:32yeah, sort of tossing to some of Pat Helland's articles, if you've
31:35seen like building on quicksand or, immutability changes everything,
31:38this idea of, you know, accountants don't use erasers, all those ideas.
31:43Yeah, exactly.
31:44And I think for web developers, this is also very intuitive, where if
31:48you build a React app, for example, and you have Some complex state
31:54that you express in react use state.
31:56And now you try to somehow do the right thing based on how the state changes
32:02using some react use effect, for example.
32:05They're like, you should use better, better mechanisms and better foundations
32:10for that, for example, using XState for like some, some state machines, et cetera.
32:15This is where you.
32:16Very explicitly and declaratively deal with the state changes as opposed to
32:21like, trying to somehow, reinterpret how some, like, nitty gritty state
32:27things have changed, whereas, like, if you just have a beautiful, simple
32:30event that is easy to understand, okay.
32:33That thing has changed.
32:34The robot has entered this room.
32:37that's much easier to understand than interpreting the
32:40coordinates of a certain thing.
32:43And this also feeds into features that you might want to give to your
32:45users, especially in productivity apps.
32:47You want to have that change history where you can see what was everyone doing.
32:51You also want to have undo's.
32:52basically what you can do for undo is when you create this action or this mutation
32:56describing, the high level intent, you can also tag along with it, a mutation saying,
33:01here's how to undo this operation later.
33:03And then you store that somewhere and then you just have a queue somewhere in
33:06your app that's like the action queue.
33:07You can go through that and undo things in a nice way.
33:11Hopefully, you know, the users will be happier with this than if you
33:14just, you know, revert states exactly, ignoring collaborators updates.
33:19Right.
33:20So, in this quadrant of the event sourcing quadrant here, there's still
33:24a couple of like sub cells, um, how the mutations are applied, namely the
33:31serializable, CRDT ish, and OT ish.
33:34Can you give a little bit of an intuition how they differ in the implementation and
33:39when you would choose one or the other?
33:41Yeah.
33:41So the examples here mostly concern text editing, which is not a coincidence.
33:45So in text editing, when you're doing any sort of collaborative text editing,
33:48like in Google Docs, you have this problem that your operation might say, I
33:52want to type, you know, the word hello.
33:55After, you know, maybe I want to type the word world after hello.
33:58So the, this message that you're going to send to the server might
34:01say something like insert world at index five, because you know, hello
34:04is five characters long, but someone else might also edit this world.
34:07Hello, or this word.
34:09Hello.
34:09Before your change makes it to the server.
34:12So maybe now it's like, hello there world.
34:15It's what you want to happen.
34:16But your edit is still trying to target index 5, so it's going to
34:19go in sort of the wrong place.
34:21You want it to shift over to accommodate edits that have been
34:24before yours in the document.
34:26And of course, this gets worse if you're, like, editing the bottom
34:28of the document, someone else is editing the paragraph on top.
34:31All of your array indices are going to get horribly messed up
34:34by the time they reach the server.
34:35Like, they're not going to be accurate anymore.
34:37So the three choices here are basically different ways to patch up those
34:41indices so that they make sense again.
34:43That makes sense.
34:44And, I think this is a common theme for local-first software is that
34:50there are a couple of like special buckets that deserve special treatment,
34:55namely text editing and also lists.
34:58And those, the, the latter two are, I think also like closely related.
35:03So on that note, the article, you also went, went a bit more in depth.
35:08on possible approaches to tame lists in this distributed setting.
35:14Will you mind sharing a little more context about that?
35:17Sure.
35:18Yeah, so if the list, as you said, it's hard and it's hard specifically
35:22because of this index problem where your obvious choice for what operations
35:25you're going to send over the network often don't make sense anymore by
35:28the time they reach the server.
35:30Um, and the solutions really fall into two camps.
35:33There's the operational transformation camp, which is used by Google Docs
35:37Which is where you're going to send, you know, index five, that sort of
35:41thing, a raw number, and the server is going to look at these, this index.
35:44It's going to look at all the intervening operations that arrived
35:48that you didn't know about, but have already reached the server.
35:50And it's going to sort of like walk through those one by one to try to
35:54figure out what index you actually meant.
35:56Because it's going to see, okay, if you inserted something at index five and
36:00three other characters have been inserted before that, I'm going to change it from
36:04five to eight, just adding five and three.
36:06Got it.
36:07So a very common app use case for this is, let's imagine Notion where
36:11on the left sidebar, you can have your, your favorite, pages pinned
36:17and those you control the order.
36:19So you can move them around or also on a Notion page, all the
36:23blocks you can reorder yourself.
36:26And a very naive approach would be, whenever you reordered something, you send
36:32to the server a full copy of the entire document, and that contains the order.
36:37But that is not very useful in the collaborative setting where
36:41now the merge radius of the entire thing is the document.
36:44And it doesn't really allow for collaboration on a per block level.
36:48And the another naive approach would be to send the block and say
36:54like, oh, now I'm at position three.
36:57But something else might've already, moved and it's no
37:00longer in reality position three.
37:02So this is what this is all about and, uh, the different approaches for this.
37:06Figma has written also a really nice blog post about this, how they,
37:11tamed this problem, where I think they call it fractional indexing.
37:14And I think you connected the dots here.
37:17can you, draw a line between the different approaches here, the CRDTish approach
37:22and the OT ish approach, and how that relates to the, the list indexing problem?
37:27Yeah.
37:27So the, the OT ish approach, that's what I was describing with, you know, you send
37:32index five to the server, but the server's going to rewrite it to index eight.
37:36So this is really this idea that the server is going to.
37:39Mutate your operation to try to make it still make sense.
37:43Then the CRDT ish approach, which is used by fractional indexing and YDS and
37:46those sort of things, is actually the clients, instead of sending, you know,
37:50index 5 to the server, they're going to rewrite this message in a way so
37:54that it still makes sense, even if it reaches the server a little bit late.
37:58So, for example, you could have, in fractional indexing, you might label
38:02your characters with these decimal numbers instead, where you say, like,
38:05the characters are at 0.1.2.3, etc.
38:09And then if you want to add a new character in between 0.
38:124 and 0.
38:135, you give it the label 0.
38:1445.
38:16So this isn't really a list index, it's what they call a fractional index.
38:19And the idea is that this will still go in between the characters at 0.
38:224 and 0.
38:225, even if some other changes happen elsewhere in the list.
38:27Because those other changes don't actually change your fractional index.
38:29You're keeping the characters at the same 0.4.5.6, etc.
38:34Right.
38:34And now the 0.
38:3545, this is what you use to derive the, the real.
38:40Integer indexes from by lexicographically ordering it.
38:45Got it.
38:45Yeah.
38:45So I'm using the same mechanism inspired by the ideas of like the, the Figma
38:51blog posts, et cetera, for Overtone.
38:53And I'm even using it before I started implementing syncing, just because
38:57I found it to be the Easiest way to keeping a list ordered in an event
39:03source system, since this is what I'm also already using to circumvent schema
39:07migrations for the, the app I'm building.
39:10So it's, I think it's actually a very simple self contained concept that can
39:15be applied even outside of the scope of a full blown local-first data stack.
39:20Yes, exactly.
39:21Yeah.
39:22It turns out so what.
39:23Text editing CRDTs are doing is very similar to factional indexing,
39:27just with some extra changes to solve some bugs, basically, like
39:30what happens if two people try to insert a character at the same place.
39:33Factional indexing breaks down, CRDTs just have the smallest change
39:36needed to make this not break down.
39:38I agree with your point that this isn't really a collaborative thing.
39:41This is just a general data structures thing.
39:43It's like the way we describe text and as an array is sort of flawed because
39:48array indexes are changing all the time, even though the character is staying
39:52the same and staying in the same place.
39:54Intuitive sense.
39:56So what we really want is an abstraction where the characters keep the same
39:59identifier at all times, whether that's a fractional index or whether it's part of
40:03the list CRDT internals, and then that's how we should represent sequences that
40:07can move around, which is basically any list in a GUI where you can drag something
40:11in between two existing elements.
40:13That makes a lot of sense.
40:14And what's also so cool about like seeing all of the different options in this
40:19classification table is that you don't have to choose exactly one for your app.
40:24what I'm planning to do for, for Overtone is mostly follow the event
40:28sourcing idea for collaborative state.
40:30However, in the places where I have.
40:33complex, particular problems such as a description text or like a document
40:39text, this is where I most likely will resort to something like Automerge
40:43or Yjs to let those technologies deal with the text editing, the
40:49collaborative text editing use case.
40:51But, and with that, I'm gonna I think I get the best of both worlds where I get
40:57all the benefits from event sourcing for the, the more high level data structure
41:02of my app and for the specificness of the text editing, I embed a little CRDT
41:08use case in the broader document use case that I tame with event sourcing.
41:14Do you think that general approach makes sense?
41:16Yes, that's exactly the way to do it.
41:18Yeah, if you look, there's a lot of, you know, blog posts saying
41:20about how CRDTs are complicated or they're hard to implement.
41:23Usually these blog posts are talking specifically about the text editing part.
41:27That's sort of the hard part where you want to let someone else do
41:29it and have their nice battle tested, fuzz tested implementation.
41:32But for other data structures, like if you have, you know, sort of a database table
41:37sort of structure or a map structure, it's easier to make your own sync engine for
41:42that and just drop in an existing library to handle the lists and text editing.
41:46Right.
41:46So it's funny that you came from like going super deep on CRDTs of like spanning
41:52this, broader table of possibilities.
41:56And it seems like now you're actually much more drawn.
41:59So the first quadrant around event sourcing, what, led
42:02to, to this interest for you?
42:05Let's see.
42:05So it might just be, you know, the grass is greener on the other side.
42:08I haven't tried to make an app or a library using the
42:11event sourcing approach yet.
42:12So maybe I just don't know what's wrong with it.
42:14but it really started out about a year ago.
42:16I was thinking about version control.
42:19This was around the same time that Ink and Switch was thinking about version
42:21control with their, upwelling essay.
42:24and the idea was like, what if we could do this Git style model where
42:28you make changes to an app, like a text document or a spreadsheet.
42:31We just put these into linear branches, and then when we merge them, you
42:35copy from one branch to another.
42:37And originally, the idea is we're going to put CRDT operations in these
42:40branches, because that's what I'm familiar with, but I eventually realized like,
42:43actually, because the branches put the operations in a total order anyway, we
42:47don't care about the CRDT correctness properties that say that you can
42:51apply operations in different orders.
42:53So we might as well just use arbitrary operations.
42:55And that unlocks a whole lot of possibilities that would have been
42:59hard to do in a CRDT system, like you can do these rename variable or find
43:03and replace operations, maybe even like a change tone with AI operation.
43:07Just put these in a log, have the log be in a fixed order, and
43:11run the operations in that order.
43:12That makes sense.
43:13so aside from the versioning use case, can you think how, using a CRDT approach
43:20versus an event sourcing approach might be a good or a bad fit for different
43:26categories of apps that you can think of?
43:29Sure.
43:29Yeah, so I think the advantages of a CRDT approach, well first off,
43:33you can do this more database model.
43:34If I'm going to put my data in a magic box that says database, and
43:37it's going to synchronize it for me, I don't have to worry about it.
43:39Whereas you're doing an event sourcing approach, you have to think
43:42more carefully about what are my mutations that I'm sending around?
43:46How do I process them?
43:47How do I make sure that they still make sense, even if someone else's
43:50mutation reach the server first?
43:52So that's a bit harder.
43:54the other advantage of CRDTs is the efficiency perspective.
43:57You can have, the CRDTs can implement operations in a very efficient way so
44:01that you're not going to accidentally say, you know, I'm sending this mutation
44:06to the server that's going to take.
44:07an entire second to process is going to slow everyone down.
44:10It's sort of the, the general trade offs that CRDTs behave more like a database.
44:14They, they just work and they're optimized to be fast.
44:17Which, with an event sourcing model, you get flexibility.
44:21You can send arbitrary mutations around, you can have arbitrary business
44:25logic on the server, it can even differ from the logic on the clients.
44:29Just coming back to the video game example, you have a lot of logic that
44:32the server needs to step through, checking permissions, checking
44:34collisions, that sort of thing.
44:36Which would be hard to do with a CRDT or with a database model.
44:40So you mentioned that you haven't yet built larger systems with the
44:45event sourcing approach, but I think you've still done a little
44:48bit of research on what might await you in the event sourcing world.
44:53So could you outline a little bit of like the potential concerns
44:57you see on the horizon when going all in on event sourcing?
45:01Yeah, so I guess the main concern always is if you're Sending around
45:06this log of events to clients.
45:09And if you're storing this as your single source of truth, then
45:13storing all these events forever, it might take up a lot of space.
45:15If you could imagine a text document, if each text character corresponds to 100
45:20bytes of JSON, then the history of all the events is going to be a hundred times
45:25bigger than the actual text document.
45:26Even if you've since cleared out the entire text document, now it's empty.
45:29You still have all this state.
45:30So that's the main challenge is just how do we store the events efficiently, how
45:35do we maybe compact them, say I don't need these events anymore, I'm going to
45:38throw them away and replace the state, while still making that play nicely
45:42with, you know, clients who have been offline for a month, that sort of thing.
45:45Which sort of mechanisms do you think will mostly help to
45:49overcome some of those issues?
45:51I'm hoping the main mechanism is just To give up, basically say text is
45:56very small for any, the main sources of lots of data in your app are
46:01blobs like images or videos, which you can put somewhere else anyway.
46:05And then for the actual event describing the fine grained changes, just store
46:08them all and it's only going to be a few megabytes per document anyway.
46:11Got it.
46:13Yeah.
46:13And I think on top of that, there's also the compaction use case.
46:17Now that I have a little bit more, insight on, on that
46:21approach with building Overtone.
46:23for example, given that everything you do within Overtone, whether it's playing
46:28a track, whether it's navigating within the app, whether it's adding a track
46:31to your playlist or follow an artist, all of those are an event and Adding
46:39a track to a playlist, there you do a lot less of those than, for example,
46:45in the background, the app auto playing the next track, which is also an event.
46:52And another kind of event is if the app tries to authenticate with a music service
46:58such as Spotify to exchange tokens, which it needs to do at least Once an hour.
47:05So it does so a little bit ahead of time.
47:07So, also when you reload the app, it needs to do that.
47:11So just by the fact by, the app running in the background over time, it Racks
47:18up quite a lot of different events.
47:21And I think they're the interesting part is the nature of the events
47:25and the nature of those events also allows for different trade offs.
47:28So me putting a track into a playlist, A, there's going to be
47:33like way fewer events of those.
47:35and it's fine to keep the entire history of this around.
47:38What's so cool about this also, the fact.
47:41That, I have this event allows me to trivially implement a feature like that.
47:46I can hover over the track and I see the information when was it added by
47:51whom was it added to, to the playlist.
47:53It also makes implementing things such as undo much easier, but the other kind
47:59of events, which might be implicit or which might just be a lot more, higher
48:05quantity, what I've seen is that, it's not as crucial to keep those events
48:11around for eternity, but some of those events are then also made irrelevant by
48:17follow up events of the, the same type.
48:20So for example, if your app has authenticated and overrides sort of like
48:24an off state into the database, and.
48:27two hours later, it has already done so 10 more times.
48:31I don't need to keep the entire history before that, maybe besides auditing
48:35reasons, so I can just at some point remove the old events, which keeps
48:41an otherwise always growing event log at a, for this given event type
48:47at a much more like constant size, which makes it much more feasible.
48:52Another thing that I, started thinking about is like, what if you have not
48:57just like one event log, but what if you have multiple event logs?
49:01And what if you have, a hierarchy of event logs?
49:04This is something that I also want to think a little bit more about, Let's
49:08say you have a, a tree of, playlists, like a, a folder of playlists.
49:13So you have a, a playlist.
49:15And that playlist could also, possibly be a folder of other playlists.
49:20So now what does the event log exist for?
49:23Does it exist for like, everything in my library?
49:26Does it exist for a broken down to.
49:30only giving information about which playlists I have, and then I need to
49:34subscribe to another playlist, but what if that playlist is a folder?
49:38So this hierarchical aspect of it, I think this will keep me busy
49:42for, for a little bit as well.
49:43Do you have thoughts on those problems?
49:46Yeah, I mean, this, the, what you're saying is really interesting.
49:48It makes me think of the problem of ephemeral presence.
49:52So, you know, in Figma, when your collaborators are moving their
49:54mouse cursors around, you can see where they're at it every time.
49:58I would imagine Figma is not actually persisting those mouse movements,
50:01it's just sending them over the usual channels so that you can see them live,
50:04but then you forget about these events because they don't matter anymore.
50:07So I wonder if you could maybe do that for a lot of the events that don't
50:11matter as much, or even in a text editor.
50:13So one thing that's really hard with a collaborative text editor is you'd like it
50:17so that whenever you press a key, that key is immediately sent to your collaborators.
50:21But if that actually creates an event that's persisted in the log, then you have
50:24this issue of, you know, 100 times as much storage as key presses, but maybe what
50:28you could say is when you press a key, that's like an ephemeral presence message.
50:32It's not actually stored, it's just sent over the same
50:34channel as the mouse movements.
50:36And this is sort of like an ephemeral mini log that's stacked on top of the actual
50:40event log, and then every 10 seconds or so you send a compacted version of
50:44like the entire sentence that the person typed as a single event, and that's
50:47what's actually stored on the backend.
50:49I wonder if that could help at all, or if this is even possible to implement.
50:52Right.
50:53I've actually implemented a small version of that already,
50:57which I call local only events.
50:59The idea of that is that, there's kind of like hierarchies of syncing as well.
51:05There's like syncing, just from the main thread to the workers thread, which is
51:11responsible for persisting the data, but also from one tab to another tab.
51:18And, those two tabs should in some regards, Converge, and in
51:23some regards, allow divergence.
51:25so for example, if you have Notion open in two tabs, you want to be able to navigate
51:32to different documents and those different tabs, but if you're in the same document,
51:36you probably want to see the same thing.
51:38So it's the same that applies to a music app.
51:41Maybe in one tab you want to have.
51:43The playback of one track and the another one, you want to not have the same
51:48playback, otherwise you hear it twice.
51:50but you want to maybe work on a playlist.
51:53And so keeping things in sync is important, but I don't want to,
51:58constantly as the playback progresses, have persistent events for this.
52:02So I try to A, have like, very Deliberately small events.
52:08And the other thing is where I have events that are broadcasted around.
52:12But, if the app reloads, it doesn't rehydrate from those.
52:16It either catches them midway or it's not important enough.
52:21that it shows it so very similar to the presence feature in Figma.
52:25So I have implemented a first version of this, but I think there can be
52:29use cases where you might want to keep them around for like 10 minutes
52:34or 10 seconds, like you say, and then have a version of compaction.
52:37I think that that's really interesting.
52:40What you're describing sounds really cool.
52:41I'll be interested to see this code someday.
52:43I'm planning to open source it a little bit further down the road.
52:47So you've now been in the local-first space for over five
52:50years, and I'm sure you've seen many technologies come along over time.
52:56I'm curious whether you have certain strong opinions about the local-first
53:00space or the web ecosystem more broadly.
53:02Yes, I guess one.
53:04Well, this isn't really an opinion, but just I'll make an observation that the
53:06local-first movement has really exploded just within the past 12 or 18 months.
53:11Like, starting out five years ago reading CRDT papers and going to CRDT
53:15conferences, it was much more, you know, mellow academic atmosphere.
53:19But now there's just so many tools popping up, I can't keep
53:21track of them in my browser tabs.
53:23you know, the local-first discord, all that stuff.
53:25Just a lot more activity.
53:26So it's both exciting and also a bit scary, because now I can't read all
53:29the papers that come out anymore.
53:31yeah, in terms of opinions, I guess the The strong opinion I've had in the
53:35past year or so is that the local-first ideal, I think, is too hard right now.
53:41There's just too many problems we'd have to solve to actually make like
53:43a local-first app where the hosting provider can go away and you'll still be
53:47able to collaborate and keep your data.
53:49So the problem that I've been focusing on for the past year is the narrow
53:53goal, like the baby step, of how do we make traditional central server SaaS
53:58collaboration easier to implement, and maybe a bit easier to deploy.
54:02So that's working on primitives like what you were describing with LiveStore.
54:05We want some way to have events that you send around and persist IndexedDB.
54:10broadcast channel between different tabs and then eventually send it
54:13to a server that stores them and broadcasts them back to the client.
54:16Just make some really good implementation of that that people can reuse so they
54:19don't have to reinvent it every time.
54:22and I think that'll be.
54:23Both useful for, you know, developers and also a good stepping stone
54:26towards the eventual goal of we want to get rid of this server and
54:29have our, have our data forever.
54:31I love that observation, and that opinion.
54:34I think that's also one of my key takeaways from talking to many folks
54:38at the local-first conference we had this year in Berlin, where Everyone
54:42gets excited about all the goals and all the ideals of local-first, but
54:48going after a few of those already is technically very complicated.
54:54And then going like all the way to making sure that the software still
54:58works if the vendor goes away, etc.
55:01That is, I think, right now achieved by only a very, very few
55:07set of products and technologies.
55:09I hope that in five years from now, it will be table stakes.
55:13But, I think it's a little bit like Maslow's hierarchy of needs.
55:17And like we, here we have like the hierarchy of ideals and we haven't, Yet
55:21quite made it as easy to achieve all of it, hopefully we'll, we'll get closer
55:26to that over the next couple of years.
55:29So those technologies that you've, now mentioned, is there anything
55:33that you're working on that can be looked at by other people?
55:37Let's see.
55:37So the main project I've had recently is it's a library called list-positions.
55:42So you can read about it on my blog post or look at the docs on GitHub.
55:45But it's basically trying to solve this fractional index generalization problem.
55:49You can think of it like a fractional index library that also
55:52implements the extra features that CRDTs have to prevent some bugs.
55:57The idea is that you can use this as a drop in part to do just the text and
56:01list collaboration in some arbitrary data structure . So I built examples on top
56:06of Triplit, Electric SQL, Replicache.
56:09So these are our collaborative data stores that don't talk
56:11about lists or texts at all.
56:13They're basically syncing maps or database tables.
56:15And I said here, if we just stick these souped up fractional
56:18indices on top, we can actually do text to rich text collaboration.
56:21So that's, that's been my focus.
56:23Very interesting.
56:24I will check this out.
56:25Maybe I can use it for Overtone.
56:27Maybe I could even integrate it with LiveStore.
56:30I will certainly check this out and we'll put the link in the show notes.
56:34Great.
56:34Matthew, is there anything else you want to share with the audience?
56:38No, I don't think so.
56:39It's been a really good chat.
56:40Thank you so much for sharing all of your knowledge about different
56:44approaches to syncing state.
56:46I think this is the most in depth we've gone on those topics so
56:49far, and it provided a brilliant overview for future conversations.
56:53Has helped me a ton to, to better understand this, both your blog
56:57posts as well as this conversation.
56:59So thank you so much for taking time today and coming on to chat.
57:03Yeah, thanks so much for having me.
57:04Thank you for listening to the local-first FM podcast.
57:07If you've enjoyed this episode and haven't done so already, please subscribe and
57:10leave a review wherever you're listening.
57:12Please also share this episode with others.
57:15Spreading the word about the podcast is a great way to
57:17support it and to keep it going.
57:19A special thanks again to Rocicorp and Expo for supporting this podcast.
57:24See you next time.