The worst enemy of any app is storing data. Well, it's also its best friend. And that's precisely what makes storage such a big hassle 🤯
Most problems derive from organizing data storage, being it database, local storage, state management. Conversely, storage is what makes an app usable, remembering things 🙌
So, what would be the ideal storage solution for a web app?
That's how my struggle evolves 👇
Remote database: complexity, complexity everywhere
Storing data in a remote database is a simple model. Single source of truth, all in one place, designed to be fast and easy to query.
It's everything that's around it that becomes bloated. Suddenly, you need a backend, and API, asynchronous request, possible failures, serialization.
The link between your device and the remote database is indirect and complex. Image from "Solving the distributed schema problem with @effect/schema by Jess Martin (Effect Days 2024)"
With 2024's technologies, this seems absurd to me. Having tasted the promise of local-first makes remote databases even more sour.
I really don't look forward for yet another complex app with remote storage 😒
Local storage is (mostly) just a cache
Don't like remote? Obvious solution: local!
Well, no.
Most local storage solutions are designed to be, well, local. Lightweight, fast, distributed. A glorious cache. That's indeed not such a robust solution for a production web app 😕
Local storage is also local, aka not shared. Want shared data? A whole new set of problems come with it 😈
Syncing is (mostly) not here yet
The advantages of local data are clear. However, we want the advantages of remote storage as well. Everything!
That's where we enter the realm of distributed systems. As hard of a problem as it gets it turns out 🥲
A solution may be a sync engine. Indeed, the idea sounds awesome: a single lightweight backend only working as a "pipe" linking multiple local devices.
Just a simple sync service in between that works with all your apps, just syncing data between local devices. Image from "The past, present, and future of local-first - Martin Kleppmann (Local-First Conf)"
Problem is we are not there yet. The web evolved as a mostly-stateless model: make a request, get response, bye-bye. Many teams working on a possible solution, but all looks still like a big work in progress.
I wrote at length about this and the whole situation with local first in a previous newsletter: The State of Local-First.
What about data import/export?
If a sync engine is not available, what about an async engine?
Instead of syncing all the data in "real time", you allow the user to export its local data, and import it back in another device.
You keep all the advantages of local-only, while preserving at least in part a resemblance to "shared data".
I have yet to explore this in practice. Writing this make it sound like not too-bad a solution, I'll see 🤔
Hybrid solution (dump the backend)
Import/export can be made easier using an in-between cloud storage.
Instead of exporting on-device and requiring the user to copy/paste/import manually, a cloud can sit in between to do this semi-automatically.
You click a button, a file is exported on your cloud provider, you open another device, click import, and select the file from the cloud.
Doesn't sound too bad, does it? I'll need to explore this as well 🤔
Performance and privacy
It's insane to me that to send a message to a friend in my neighborhood, my message must first hit a server on the other side of the world.
It's an issue of performance, but also privacy and ownership. I want my apps to store my ideas and information, why is this required to be tied to a server?
It's not, or at least that's the direction I am heading to.
New release coming: Effect with React 19: Project Template 🚀
I packed all my most common practices while working with Effect in React 👇
New React 19 + @EffectTS_ project on Typeonce coming 🔜 I have been using Effect with React for a while now In this template I walk you through how I organize a client/server react project with Effect 👇 typeonce.dev/course/effect-…
I am also planning an update to Typeonce's landing page (updated offer). This may interest you, stay tuned 👀
See you next 👋