@spinningideas/client-store v0.3.15

client-store

A simple data storage library primarily design to work in a web based application environment that uses a modern web browser. It uses localStorage, while in Node.js it attempts to provide a compatible in-memory implementation. It provides a set of functions to store structured data like a database containing tables and supporting queries and standard CRUD operations for data. It provides basic insert/update/delete/query capabilities similar to a database and extend what is available in localStorage. The structured data is stored as serialized JSON in the selected storage engine.

WARNING (Alpha Version)

This code is in active development and should not yet be used in production. The API is subject to change. There is ideation and work in progress to extend the possible storage engines to include IndexedDB and other storage engines including in-memory storage and sync to remote storage engines. This was part of the driver for the fork of localStorageDB.

License

• Licensed: MIT license

Credits and Inspiration

• localStorageDB This package was forked from localStorageDB which was created by Kailash Nadh (https://github.com/knadh/localStorageDB) and the code was updated to modern JavaScript standards and TypeScript support added and naming changed except for the public methods. Many thanks to Kailash Nadh for the original implementation and inspiration. The package appears to no longer be maintained and projects that depended on the package needed to evolve in breaking fashion with new features. The original forked code that was the basis for this package is available in this repo under src folder in the localStorageDB.ts file and this code will be used as a reference for future development and will NOT be altered. - Initial changes (made in new code forked from localStorageDB.ts into clientStore.ts) include: - Updated to modern JavaScript standards (ES6+, parameters pascalCase etc) - Added JSDoc comments - Renamed ID record identifier from "ID" to "ROW_IDENTIFIER" to be more consistent with SQL naming conventions and change datatype from integer to uuid - Change internal methods and structure in preparation for adding support for other storage engines and doing more work
• IndexedDB
• https://github.com/dreamsavior/Better-localStorage
• lowdb
• local-storage-db
• use-local-storage
• convex-backend
• clientdb
• tiny-localstorage-db
• sqlite3
• localForage - https://github.com/localForage/localForage/blob/master/src/localforage.js

Dependencies

The package has no dependencies other than the browser's localStorage APIs. There is additional code to support situations that do not have access to the browser's localStorage APIs. This includes Node.js environments and in this case the package uses global.clientStoreMemoryStorage as an optional dependency for Node.js environments.

NOTE: The maximum storage capacity for localStorage varies across browsers, but a common limit is around 5-10 MB per origin. Some browsers, like Safari, may prompt the user to increase the limit if the initial quota is exceeded, while others may silently fail to store data if the limit is reached.

• localStorage
• node-localstorage - this can be used in a node environment to provide a compatible in-memory storage implementation.

Features

• Provides ability to store (on the client-side withing web based applications) structured data like a database containing tables and rows of data in a tabular format.
• Supports query operations against the stored data and standard CRUD operations (basic create/read(query)/update/delete capabilities).
• The structured data is stored as serialized JSON in the selected storage engine (localStorage, node-localstorage, or a custom storage engine).

Feature Roadmap

1. Add tests for the pagingation and sorting features in query.
2. Implement support to store JSON objects and arrays as column values.
3. Create a GitHub Actions workflow to automatically run these tests on pull requests
4. Add GitHub action to automatically build and publish the package to npm on push to main
5. Add GitHub Pages to host the documentation
6. Add support for IndexedDB - see https://github.com/localForage/localForage/blob/master/src/drivers/indexeddb.js
7. Add support for SQLite
8. Add support for remote storage engines and support for sync to remote storage engines

Installation

NPM

npm install @spinningideas/client-store

Run Tests

See testing section below in the README for more information on running tests.

npm run test

Supported Environments

Browsers

Browsers need to support "Local Storage" and "Session Storage" in order for clientStore to function in browser environments.

Node.js

In Node.js environments, the library provides a compatible in-memory storage implementation by default via global.clientStoreMemoryStorage, or you can use a npm storage implementation like node-localstorage.

Usage / Examples

Browser Environment

Creating a database, table, and populating the table

// Initialize. If the storage doesn't exist, it is created
const moviesStore = clientStore("movies", localStorage);

// Or Check if the table exists and if not create the table then setup data. Useful for initial storage setup
if (moviesStore.tableExists("movies") === false) {
  // create the "movies" table
  moviesStore.createTable("movies", [
    "episodeId",
    "title",
    "releaseYear",
    "boxOffice",
    "isBest",
  ]);

  // insert some data
  moviesStore.insert("movies", {
    episodeId: "IV",
    title: "Star Wars: A New Hope",
    releaseYear: 1977,
    boxOffice: 775.4, // box office in millions of dollars
    isBest: false,
  });
  moviesStore.insert("movies", {
    episodeId: "V",
    title: "Star Wars: The Empire Strikes Back",
    releaseYear: 1980,
    boxOffice: 538.4, // box office in millions of dollars
    isBest: true, // The Empire Strikes Back is considered the best
  });
  moviesStore.insert("movies", {
    episodeId: "VI",
    title: "Star Wars: Return of the Jedi",
    releaseYear: 1983,
    boxOffice: 475.1, // box office in millions of dollars
    isBest: false,
  });

  // save the data to localStorage
  // all create/drop/insert/update/delete operations should be committed
  moviesStore.commit();
}

Create and seed sata into Table in one process

// Seed rows of data for pre-population and setup of storage. Useful for initial storage setup
const rows = [
  {
    episodeId: "IV",
    title: "Star Wars: A New Hope",
    releaseYear: 1977,
    boxOffice: 775.4, // box office in millions of dollars
    isBest: false,
  },
  {
    episodeId: "V",
    title: "Star Wars: The Empire Strikes Back",
    releaseYear: 1980,
    boxOffice: 538.4, // box office in millions of dollars
    isBest: true, // The Empire Strikes Back is considered the best
  },
  {
    episodeId: "VI",
    title: "Star Wars: Return of the Jedi",
    releaseYear: 1983,
    boxOffice: 475.1, // box office in millions of dollars
    isBest: false,
  },
];

// create the table and insert records in one go
moviesStore.createTableWithData("movies", rows);

moviesStore.commit();

Alter existing Table to add two new Columns

// If database already exists, and want to alter existing tables
if (!moviesStore.columnExists("movies", "runTime")) {
  moviesStore.alterTable("movies", "runTime", 121);
  moviesStore.commit(); // commit the new columns to storage
}

// Multiple columns can also added at once
if (
  !(
    moviesStore.columnExists("movies", "runTime") &&
    moviesStore.columnExists("movies", "rating")
  )
) {
  moviesStore.alterTable("movies", ["runTime", "rating"], {
    runTime: 121,
    rating: "PG",
  });
  moviesStore.commit(); // commit the new columns to storage
}

Querying Data

// Define query parameters
const queryParams = {
  query: { releaseYear: 1980 },
};

// Simple select queries
const movies1980 = moviesStore.query("movies", queryParams);

// Query with multiple conditions
const specificMovie = moviesStore.query("movies", {
  query: { releaseYear: 1977, boxOffice: 775.4 },
});

// Select all movies (no query parameters)
const allMovies = moviesStore.query("movies");

// Select all movies released after 1979 using a filter function
const newerMovies = moviesStore.query("movies", {
  query: (row) => {
    // The callback function is applied to every row in the table
    if (row.releaseYear > 1979) {
      // If it returns true, the row is selected
      return true;
    } else {
      return false;
    }
  },
});

// Or with a more concise arrow function
const newerMoviesAlt = moviesStore.query("movies", {
  query: (row) => row.releaseYear > 1979,
});

// Select movies with box office over 500 million, limited to 2 results
const highGrossing = moviesStore.query("movies", {
  query: (row) => row.boxOffice > 500,
  limit: 2,
});

// Select the best movie (using the boolean field)
const bestMovie = moviesStore.query("movies", {
  query: { isBest: true },
});

Sorting Data

// Select 2 rows sorted in ascending order by boxOffice
const sortedMovies = moviesStore.query("movies", {
  limit: 2,
  sort: [["boxOffice", "ASC"]],
});

// Select all rows first sorted in ascending order by boxOffice, and then, in descending, by releaseYear
const multiSortedMovies = moviesStore.query("movies", {
  sort: [
    ["boxOffice", "ASC"],
    ["releaseYear", "DESC"],
  ],
});

// Combine query, limit and sort
const filteredSortedMovies = moviesStore.query("movies", {
  query: { releaseYear: 1980 },
  limit: 1,
  sort: [["boxOffice", "ASC"]],
});

Getting Distinct rows of data

// Get records with distinct releaseYear and boxOffice values
const distinctMovies = moviesStore.query("movies", {
  distinct: ["releaseYear", "boxOffice"],
});

Example Query Results

// Query results are returned as arrays of object literals
// A "ROW_IDENTIFIER" field with the internal auto-incremented identifier of the row is also included
// Thus, ROW_IDENTIFIER is a reserved field name

const bestMovie = moviesStore.query("movies", { query: { isBest: true } });
console.log(bestMovie);

/* Results:
[
  {
    ROW_IDENTIFIER: "9c96a0e5-c372-45b6-8456-77e24720fa56",
    episodeId: "V",
    title: "Star Wars: The Empire Strikes Back",
    releaseYear: 1980,
    boxOffice: 538.4,
    isBest: true
  }
]
*/

Updating Data

// Update all movies from 1977 to $800M box office
const updatedCount1 = moviesStore.update(
  "movies",
  { releaseYear: 1977 },
  (row) => {
    return { boxOffice: 800.0 };
  }
);
console.log(`Updated ${updatedCount1} records`);

// Or update all movies released before 1980 to $800M box office
const updatedCount2 = moviesStore.update(
  "movies",
  (row) => row.releaseYear < 1980, // Simplified arrow function with implicit return
  (row) => ({ boxOffice: 800.0 }) // Arrow function with implicit return of object
);
console.log(`Updated ${updatedCount2} records`);

// Don't forget to commit changes
moviesStore.commit();

Upsert - Insert or Update conditionally

// If there's a movie with episodeId VI, update it, or insert it as a new row
const result = moviesStore.upsert(
  "movies",
  { episodeId: "VI" },
  {
    episodeId: "VI",
    title: "Star Wars: Return of the Jedi",
    releaseYear: 1983,
    boxOffice: 500.5, // box office in millions of dollars
    isBest: false,
  }
);

// You can also use upsertOrUpdate which is an alias for upsert
const result2 = moviesStore.upsertOrUpdate(
  "movies",
  { episodeId: "VII" },
  {
    episodeId: "VII",
    title: "Star Wars: The Force Awakens",
    releaseYear: 2015,
    boxOffice: 2068.0,
    isBest: false,
  }
);

// If result is null, insertion failed
// If result is an array, it contains the ROW_IDENTIFIERs of updated rows
console.log(result ? `Updated ${result.length} rows` : "Inserted new row");
console.log(result2 ? `Updated ${result2.length} rows` : "Inserted new row");

moviesStore.commit();

Node.js Environment

Using the built-in in-memory storage

// Import the clientStore module
const clientStore = require("@spinningideas/client-store").default;

// Initialize with default in-memory storage (no second parameter needed)
const userStore = clientStore("users");

// Create a table and add data
userStore.createTable("users", ["id", "name", "email", "active"]);

userStore.insert("users", {
  id: 1,
  name: "Luke Skywalker",
  email: "luke@example.com",
  active: true,
});

userStore.insert("users", {
  id: 2,
  name: "Yoda",
  email: "yoda@example.com",
  active: false,
});

// Commit changes to the in-memory storage
userStore.commit();

// Query the data
const activeUsers = userStore.query("users", { query: { active: true } });
console.log(activeUsers);

Using a third-party storage implementation

// Import the clientStore module and a third-party storage implementation
const clientStore = require("@spinningideas/client-store").default;
const nodeLocalStorage = require("node-localstorage");

// Initialize with the third-party storage implementation
const configStore = clientStore("app-config", nodeLocalStorage);

// Create a table and add data
configStore.createTable("settings", ["key", "value", "description"]);

configStore.insert("settings", {
  key: "theme",
  value: "dark",
  description: "UI theme preference",
});

configStore.insert("settings", {
  key: "notifications",
  value: "enabled",
  description: "Notification settings",
});

// Commit changes
configStore.commit();

// Query the data
const themeSettings = configStore.query("settings", {
  query: { key: "theme" },
});
console.log(themeSettings);

Using the ClientStorage class directly

// Import the clientStore module and ClientStorage class
const {
  default: clientStore,
  ClientStorage,
} = require("@spinningideas/client-store");

// Create a custom storage implementation
const myCustomStorage = new ClientStorage();

// Initialize with the custom storage
const dataStore = clientStore("custom-data", myCustomStorage);

// Use the store as normal
dataStore.createTable("items", ["id", "name", "category"]);
dataStore.insert("items", { id: 1, name: "Item 1", category: "A" });
dataStore.commit();

const items = dataStore.query("items");
console.log(items);

Deleting Data

// Delete all movies from 1977
const deletedCount1 = moviesStore.deleteRows("movies", { releaseYear: 1977 });
console.log(`Deleted ${deletedCount1} records`);

// Delete all movies published before 1980
const deletedCount2 = moviesStore.deleteRows(
  "movies",
  (row) => row.releaseYear < 1980
);
console.log(`Deleted ${deletedCount2} records`);

// Commit the deletions to localStorage
moviesStore.commit();

Available Methods

Storing complex objects

While the library is meant for storing fundamental types (strings, numbers, bools), it is possible to store object literals and arrays as column values, with certain caveats. Some comparison queries, distinct etc. may not work. In addition, if you retrieve a stored array in a query result and modify its values in place, these changes will persist throughout further queries until the page is refreshed. This is because clientStore loads and unserializes data and keeps it in memory in a global pool until the page is refreshed, and arrays and objects returned in results are passed by reference.

If you really need to store arrays and objects, you should implement a deep-copy function through which you pass the results before manipulation.

Package Publishing

This package is set up to be published to npm with support for both CommonJS and ES Modules. The package includes the following features:

• CommonJS build for Node.js and legacy environments
• ES Modules build for modern bundlers and environments
• TypeScript declaration files
• Tree-shakable exports

Using the Package

ES Modules (recommended)

import clientStore from "@spinningideas/client-store";

const store = new clientStore("myDatabase");

CommonJS

const clientStore = require("client-store").default;

const store = new clientStore("myDatabase");

Publishing to npm

To publish a new version of the package to npm:

1. Update the version in package.json
2. Run tests to ensure everything is working correctly: npm test
3. Build the package: npm run build
4. Publish to npm: npm publish

Alternatively, you can use npm version commands which will handle versioning and tagging:

npm version patch  # for bug fixes
npm version minor  # for new features
npm version major  # for breaking changes
npm publish

Testing

Running Tests

The client-store library uses Mocha and Chai for testing. To run the tests, use the following command:

npm run test

Test Coverage

To run tests with coverage reporting, use:

npm run test:coverage

This will run the tests and display a coverage summary in the terminal, showing:

• Percentage of statements covered
• Percentage of branches covered
• Percentage of functions covered
• Percentage of lines covered

Coverage Report

To view a detailed HTML coverage report in your browser, run:

npm run coverage:report

The HTML report provides a detailed view of which parts of your code are covered by tests and which aren't, with color-coded highlighting:

• Green: Code that is covered by tests
• Red: Code that is not covered by tests
• Yellow: Branches that are partially covered

Test Structure

The tests are organized into several sections:

1. Basic Operations: Tests for core functionality like creating stores and tables
2. Storage Operations: Tests for storage-related methods like import/export and storage management
3. CRUD Operations: Tests for data manipulation methods (Create, Read, Update, Delete)
4. Additional Table Operations: Tests for table-specific operations
5. Error Handling: Tests for proper error handling in various scenarios

Adding New Tests

When adding new functionality to the library, please ensure you add corresponding tests to maintain good test coverage. Tests should be added to the appropriate section in tests/client-store-tests.js.

npm package publishing

Pre-requisites

npm pack npm version minor npm publish