Local Storage Database with Typescript

Local storage is a great solution for quick prototyping.

Designing a complete database schema and hooking my baby project to a remote database is too much work. I don't even know if this project will last for the next 2 days. Why should I spend 3 hours setting up a backend database?

At the same time, local storage is often a pain.

I want an SQL-like experience, schema validation, and a simple API to write, read, and delete from local storage

Here you can find my own generic implementation of a local storage database API. It comes with the following features out of the box:

Powerful schema validation (using zod)
Error handling and functional programming (using fp-ts)
Simple API to write, read, update, and delete in local storage

Prerequisites

This implementation uses Typescript to define a type-safe interface for our database.

Furthermore, we are also using the following packages, make sure to install them:

zod: Schema validation, make sure that the data inside the database has the right format
fp-ts: Functional programming types, used to provide a type-safe and composable interface for the database API

Install them all using the following command:

npm install zod fp-ts

Generic interface for any storage implementation (`IStorageApi`)

Below you can find the implementation of a generic type for a complete API that defines all the methods necessary to interact with your database.

Every concrete implementation of our database will be of type IStorageApi:

Schema: zod schema used for validation
Error: any kind of error returned by the API (using Either from fp-ts)
Table: defined the "tables" of our database. This makes possible to use the same API for multiple storages. It works similar to a key for local storage

istorage.api.ts

import * as RTE from "fp-ts/ReaderTaskEither";
import { z } from "zod";

type IStorageApi<Schema extends z.ZodTypeAny, Error, Table extends string> = (
  schema: Schema
) => {
  /**
   * Given a `Table`, return all the data inside storage.
   */
  readAll: RTE.ReaderTaskEither<Table, Error, z.output<Schema>[]>;

  /**
   * Read all the data inside `Table` filtered by the `check` function.
   */
  readWhere: RTE.ReaderTaskEither<
    [Table, (check: z.output<Schema>) => boolean],
    Error,
    z.output<Schema>[]
  >;

  /**
   * Given a `Table` and a value, write the value inside storage (single value).
   */
  write: RTE.ReaderTaskEither<
    [Table, z.output<Schema>],
    Error,
    z.output<Schema>
  >;

  /**
   * Given a `Table` and a list of values, write all the values inside storage.
   */
  writeAll: RTE.ReaderTaskEither<
    [Table, z.output<Schema>[]],
    Error,
    readonly z.output<Schema>[]
  >;

  /**
   * Delete all the data inside `Table`.
   */
  deleteAll: RTE.ReaderTaskEither<Table, Error, unknown>;

  /**
   * Update all the data inside the given `Table` based on the
   * given `check` function (**map**).
   */
  update: RTE.ReaderTaskEither<
    [Table, (check: z.output<Schema>) => z.output<Schema>],
    Error,
    readonly z.output<Schema>[]
  >;
};

export type { IStorageApi };

Local storage database implementation

All we need to do now is to implement the IStorageApi interface for localStorage.

Here below the complete implementation:

local-storage.api.ts

import * as A from "fp-ts/Array";
import * as E from "fp-ts/Either";
import { pipe } from "fp-ts/function";
import * as IOE from "fp-ts/IOEither";
import * as TE from "fp-ts/TaskEither";
import { z } from "zod";
import { IStorageApi } from "./istorage.api";

type LocalStorageApi<
  Schema extends z.ZodTypeAny,
  Table extends string
> = IStorageApi<Schema, string, Table>;

type LocalStorageApiSchema<
  Schema extends z.ZodTypeAny,
  Table extends string
> = Parameters<LocalStorageApi<Schema, Table>>[0];

type LocalStorageApiMethod<
  Schema extends z.ZodTypeAny,
  Table extends string
> = ReturnType<LocalStorageApi<Schema, Table>>;

type LocalStorageApiData<
  Schema extends z.ZodTypeAny,
  Table extends string
> = z.output<LocalStorageApiSchema<Schema, Table>>;

const readAll =
  <Schema extends z.ZodTypeAny, Table extends string>(
    validation: Schema
  ): LocalStorageApiMethod<Schema, Table>["readAll"] =>
  (table) =>
    pipe(
      () =>
        E.tryCatch(
          () => localStorage.getItem(table),
          () => "Error when loading from local storage"
        ),
      IOE.chain((item) =>
        item === null
          ? IOE.fromEither(E.of<string, z.output<typeof validation>[]>([]))
          : pipe(
              () =>
                E.tryCatch(
                  () => JSON.parse(item) as unknown,
                  () => "Error when parsing to JSON"
                ),
              IOE.chain((json) =>
                pipe(
                  z.array(validation).safeParse(json),
                  (parsed): E.Either<string, z.output<typeof validation>[]> =>
                    parsed.success
                      ? E.of(parsed.data)
                      : E.left(
                          `Error when parsing local data: ${parsed.error.issues[0].message}`
                        ),
                  IOE.fromEither
                )
              )
            )
      ),
      TE.fromIOEither
    );

const readWhere =
  <Schema extends z.ZodTypeAny, Table extends string>(
    validation: Schema
  ): LocalStorageApiMethod<Schema, Table>["readWhere"] =>
  ([table, check]) =>
    pipe(table, readAll(validation), TE.map(A.filter(check)));

const write =
  <Schema extends z.ZodTypeAny, Table extends string>(
    validation: Schema
  ): LocalStorageApiMethod<Schema, Table>["write"] =>
  ([table, item]) =>
    pipe(
      validation.safeParse(item),
      (parsed): E.Either<string, z.output<Schema>> =>
        parsed.success
          ? E.of(parsed.data)
          : E.left(
              `Invalid schema for writing data: ${parsed.error.issues[0].message}`
            ),
      TE.fromEither,
      TE.chain((validItem) =>
        pipe(table, readAll(validation), TE.map(A.append(validItem)))
      ),
      TE.chain((newData) =>
        pipe(
          () =>
            E.tryCatch(
              () => localStorage.setItem(table, JSON.stringify(newData)),
              () => "Error while saving data"
            ),
          TE.fromIOEither,
          TE.map(() => item)
        )
      )
    );

const writeAll =
  <Schema extends z.ZodTypeAny, Table extends string>(
    validation: Schema
  ): LocalStorageApiMethod<Schema, Table>["writeAll"] =>
  ([table, items]) =>
    pipe(
      table,
      readAll(validation),
      TE.map(A.concat(items)),
      TE.chain((newData) =>
        pipe(
          () =>
            E.tryCatch(
              () => localStorage.setItem(table, JSON.stringify(newData)),
              () => "Error while saving data"
            ),
          TE.fromIOEither,
          TE.map(() => items)
        )
      )
    );

const deleteAll =
  <Schema extends z.ZodTypeAny, Table extends string>(): LocalStorageApiMethod<
    Schema,
    Table
  >["deleteAll"] =>
  (table) =>
    pipe(
      () =>
        E.tryCatch(
          () => localStorage.removeItem(table),
          () => `Error while deleting all storage in '${table}' schema`
        ),
      TE.fromIOEither
    );

const update =
  <Schema extends z.ZodTypeAny, Table extends string>(
    validation: Schema
  ): LocalStorageApiMethod<Schema, Table>["update"] =>
  ([table, check]) =>
    pipe(
      table,
      readAll(validation),
      TE.map(A.map(check)),
      TE.chain((newData) =>
        pipe(
          table,
          deleteAll(),
          TE.chain(() => writeAll(validation)([table, newData]))
        )
      )
    );

const localStorageApi = <Schema extends z.ZodTypeAny, Table extends string>(
  schema: Schema
): LocalStorageApiMethod<Schema, Table> => ({
  readAll: readAll(schema),
  readWhere: readWhere(schema),
  write: write(schema),
  update: update(schema),
  writeAll: writeAll(schema),
  deleteAll: deleteAll(),
});

export type { LocalStorageApiData };
export { localStorageApi };

Type helpers

Initially we define some types used in our implementation.

All these types derive from the IStorageApi interface.

Notice that we leave the Schema as generic. By doing this, we can use this same API for different schemas.

These types are all internal. The only exported type is LocalStorageApiData, which represents the shape of the database schema.

API implementation

Next we define the actual implementation of our local storage API.

Notice how each method is implemented in a separate function, instead of all together in the final localStorageApi. This allows to use different functions inside each other (for example, we are using readAll inside many of the other functions)

Some main points to highlight:

We consider any access to localStorage as potentially unsafe. For this reason, any request to localStorage is wrapped inside IOEither
We used zod's safeParse method to validate the data. In this implementation we return an error if the shape of the data is invalid (both for read and write operations). This means that if the any value in local storage gets corrupted (i.e. wrong format), then all reads operation will fail!
We are required to read all the data from local storage at every request, even when we then filter the result. This is not the most efficient solution, but hey, it's local storage 💁🏼‍♂️
Error is defined as string, and it's not left generic like Schema
We use JSON.parse and JSON.stringify to read and write data in local storage
Even if reading from local storage is a synchronous operation, the API returns TaskEither (asynchronous)

We then define a localStorageApi, derived from IStorageApi, and we export it.

How to use the local storage database

We are now free to define all the implementation that we need on top of this API.

Below an example of a concrete implementation:

user.storage.api.ts

import { z } from "zod";
import { localStorageApi } from "../local-storage.api";

const schema = z.object({
  age: z
    .number()
    .min(0, {
      message: "No human has less than 0 years, or not?",
    })
    .max(120, {
      message: "You are too old for this app",
    }),

  link: z
    .string()
    .url({
      message: "Only valid URLs allowed here (or none)",
    })
    .optional(),
});

const userStorageApi = localStorageApi<typeof schema, "User">(schema);
type UserStorageApiData = z.output<typeof schema>;

export type { UserStorageApiData };
export { userStorageApi };

As easy as it gets 🌈

Now you can simply use the API in you app:

Have fun, and good prototyping 👋