Event sourcing, local-first, and distributed state

In the rabbit hole of local-first I found a new (possible) ally: Event Sourcing.

It claims to be the solution for distributed state. And indeed it looks promising, even if it hides a lot of complexity once you take a closer look.

Never heard about it before? That's what I can tell you about Event Sourcing 🤔

How did it come to this?

I envision a future where apps are all local.

I own all my data, stored all on my device, fast, private, and secure. But that's not enough. I want to use the same data on multiple devices. The best of everything 🤝

I also secretly don't want to deal with could providers, server, Kubernetes, and those folks 🙌

The local-first movement is trying to make this a reality. So I jumped on 🚀

It's all a distributed system

Now, for the technical stuffs.

Since all the data originates locally, each device has its own database and makes its own changes (even offline).

This is level 1: a sort of "local-only", which can be easily achieved today (e.g. PGLite).

The hard part is multiplayer. Each device is doing its own things, but then there must be a way to synchronize the data. The requirement is that the final state should be eventually the same on all devices (eventual consistency).

And that's the hard part, and where Event Sourcing claims to be the solution.

There is no current state

Traditional databases store a picture of the current state of the data. That works when there is only one central authority.

The model breaks when each client dictates its own "current state". If anyone can change anything, putting together a consistent screenshot at the end becomes "chaotic" 🤯

What you would want instead is a list of all the changes, and a set of rule to apply them in the right order.

Welcome Event Sourcing 🫡

A model for local-first

Here is where I am with my current Event Sourcing model:

Each device has its own database
Each operation (1) updates the database and (2) adds a new entry to an event log
The client only cares about its own database (reactive queries)
During syncing the client sends the event log to the server
A (lightweight) server sends the event log to other clients
Each client applies the events from the server in the right order, updating its database

As long as the order is guaranteed between clients (e.g. HLC) each client should eventually end up with the same database.

It works, but

The model is simple enough, too simple I suspect. I still need to try to put this into code before discovering the limitations.

Key problem: What should the server send to each client? The full event log? How can a client avoid applying the same events twice?

I am also looking into a way to make it "real" local-first 👇

In local-first software, the availability of another computer should never prevent you from working:

Multi-device/Multiplayer (not local-only)

Offline support

It works even if the app developer goes out of business and shuts down the servers

My ultimate goal is to have a shared server that handles synchronization of events between any clients and multiple apps.

In practice, it would mean making the server a singleton, and only focus on beautiful and fast client code and UIs.

Eventually 🫡

Meanwhile some cool updates on Typeonce. I figured that the best way to show the "updated date" of a course is to display the versions of the packages used. Done 👍