@wessberg/syncdb NPM

SyncDB

An offline-first, real-time, observable, decoupled, client-to-client distributed database.

Installation

Simply do: npm install @wessberg/syncdb.

What is it

This is a Offline-First, real-time, observable, decoupled client-to-client distributed database. It uses a central server to propagate messages around between clients, but it has no centralized data store. This means that the data in the network at all times reflects the contents of the local databases of the peers, and thus provides fully decentralized storage.

It is a specialization of LocalDB, which is a client-side, observable database based on IndexedDB. The key difference is that SyncDB not only performs these operations but also synchronizes with remote clients by using WebSockets and real-time notifications when something changes.

Disclaimer

It is still very early days for the synchronization flow between the clients and the server. It doesn't tackle failure resilience or scenarios where a peer simply doesn't reply. For now, use this library with caution.

Usage

For the basic storage API, please refer to the documentation for LocalDB.

This document will only cover what's unique to SyncDB.

Prerequisites

You must have a SyncDBServer instance running and listening on the host and port you want all clients to communicate with each other through. This is as easy as doing a npm install @wessberg/syncdb-server and initializing an instance of it on the machine or the environment in which it can serve that host and port.

Initialization

Start by initializing SyncDB:

const db = new SyncDB({
  databaseName: "my_database",
  url: "localhost:8000"
});

// You can also explicitly define which implementation of WebSocket and URL to use by passing that on to the constructor.
// This is useful for usage in situations where you want to use a custom WebSocket implementation such as Node, where you don't have the browser's WebSocket implementation.
const db = new SyncDB({
  databaseName: "my_database",
  url: "localhost:8000",
  webSocketImplementation: WebSocket,
  urlImplementation: URL
});

Now, SyncDB will attempt to connect to a SyncDBServer instance listening on the given host and port.

WSS (Secure WebSocket Server connection)

We use the URL that you provide the SyncDB constructor to check if you want to connect to a unsecure or secure WebSocket. You can explicitly prepend the url with "wss://" or "https://" to make it connect to a secure WebSocket:

const db = new SyncDB({
  databaseName: "my_database",
  url: "wss://mydomain.com:8000/foo/bar"
});

That's it! SyncDB will automatically synchronize changes, whether coming from your local database or a remote client. You can explicitly call push() or pull() if you like, but otherwise you can just use the basic API of LocalDB. For full documentation, I'd suggest that you read the please refer to the documentation for LocalDB.

Synchronization model

All local changes are carried out immediately. After performing the local change and resolving the Promise to the client, SyncDB will carry on informing remote peers about the change. This is also called optimistic concurrency. If at least one remote client rejects the change, the local database will be informed and revert the local changes.

When SyncDB receives remote changes or explicitly pulls from a remote peer, it will update the relevant local contents if and only if no later local operations has been performed that would invalidate the remote ones.

SyncDB strives to guarantee Eventual Consistency and are optimized for speed.

Examples

const db = new SyncDB({
  databaseName: "my_database",
  url: "wss://localhost:8000"
})

interface Todo {
	text: string;
	done: boolean;
	importance: number;
}

// Adding an item to the database.
const todo = await db.add<Todo>("todos", {
	text: "write README",
	done: false,
	importance: 2
});

// Updating an item in the database.
await db.put<Todo>("todos", todo.id, {...todo, done: true});

// Remove an item from the database.
await db.remove("todos", todo.id);

// Returns true if the database has an item in the "todos" collection with a key of "123456".
await db.has("todos", "123456");

// Returns all todos that has a falsy 'done' value.
const undoneTodos = await db.filter<Todo>("todos", todo => !todo.done);

// Returns the first item that has a text value of "I <3 pasta". In this example, the cursor goes in a descending direction (from newest to oldest inserted records).
const favoriteTodo = await db.find<Todo>("todos", todo => todo.text === "I <3 pasta", CursorDirectionKind.DESCENDING);

// Maps all todos and returns an array of integers, reflecting their importance.
const importances = await db.map<Todo>("todos", todo => todo.importance);

Changelog:

v0.0.9:

Bumped dependencies.

v0.0.8:

Allowed calling 'set' without explicitly adding the store first. SyncDB will do that automatically.

v0.0.7:

Bumped dependencies.

v0.0.6:

Bumped dependencies.

v0.0.5:

Bumped dependencies and updated API to reflect changes.

v0.0.4:

Added syncing for changes of kind 'CLEAR_STORE'.

v0.0.3:

Bumped dependency on LocalDB.

v0.0.2:

Renamed 'skipSync' argument to 'isPrivate' to better reflect the fact that giving it a value of true will keep the operation private to the local client. Added signatures to interface.

v0.0.1: