0.5.1 • Published 8 months ago

@phorbas/store v0.5.1

Weekly downloads
1
License
BSD-2-Clause
Repository
github
Last release
8 months ago

PHORBAS Store

A PHORBAS store is an associative 1:1 key to binary content data storage abstraction. Key must be a string, although often hex encoded from cryptographic hashing via PHORBAS Opaque. Content is binary (ArrayBuffer), direclty supporting encrypted content. The store handles the principles of Persistent, Binary content, Addressable, and Store.

API & Use

import {kbc_js_map} from '@phorbas/store'

// use one of the many backend storage adapters
let stg = kbc_js_map()

// keys are opaque strings; some backend adapters may route based on parts of keys
let some_key = `some_key_${Date.now().toString(36)}`

await stg.bkc_exists(some_key) // 0

// store binary content addressed by key
let binary_body = crypto.getRandomValues(new Uint32Array(1024))
await stg.bkc_store(some_key, binary_body)


// key now exists
await stg.bkc_exists(some_key) // 1

// and we can fetch the binary content, returned in an ArrayBuffer
let body_result = await stg.bkc_fetch(some_key)

The backend storage API is fundamentally asynchronous, designed for use with @phorbas/opaque. The kbc prefix is for "keyed binary content".

  • async kbc_exists(key) returns 1 if exists, otherwise 0
  • async kbc_fetch(key) returns ArrayBuffer if exists, otherwise null|undefined
  • async kbc_store(key, body) returns error | null, where error is null upon success.

  • async * kbc_stream_exists(key_aiter, {signal}) is an async iterable streaming version of kbc_stream_exists

  • async * kbc_stream_fetch(key_aiter, {signal}) is an async iterable streaming version of kbc_stream_fetch
  • async * kbc_stream_store(key_body_aiter, {signal}) is an async iterable streaming version of kbc_store

Storage Implementations

JavaScript, Browser, and WinterCG

Unit TestInteg TestTechnology / LibraryImplementation
JS Map new Map()impl: code/js_map.jsy test_js_map.jsy integ--core/integ-core.test.mjs
JS Fetch API globalThis.fetch()impl: code/web/fetch.jsy integ_fetch.jsy integ--core/integ-web.test.mjs
Browser DOM Storage globalThis.localStorage or globalThis.sessionStorageimpl: code/web/web_storage.jsy test_web_storage.jsy
Browser DOM Element document.createElement()impl: code/web/web_dom.jsy test_web_dom.jsy
Browser IndexedDB API globalThis.indexedDB.open()impl: code/web/web_db.jsy test_web_db.jsy
Browser Service Worker Cache API globalThis.caches.open()impl: code/web/web_cache.jsy test_web_cache.jsy integ_web_cache_fetch.jsy

Local

Unit TestInteg TestTechnology / LibraryImplementation
LMDB node-lmdbimpl: code/local/lmdb.jsy integ--level/integ-lmdb.test.mjs
LevelDB Level ecosystemimpl: code/adapter/level.jsy integ--level/integ-level.test.mjs
Node FS Promises API import * as fsp from 'node:fs/promises'impl: code/local/fsp.jsy test_local_fsp.jsy integ--local/integ-fs-promises.test.mjs
Node FS API import * as fs from 'node:fs'impl: code/local/fs.jsy test_local_fs.jsy integ--local/integ-fs.test.mjs
Web FS & FS Promises via lightning-fs from isomorphic-gitimpl: code/local/fsp.jsy integ_web_lightning-fs.js

NoSQL

Integ TestTechnology / LibraryImplementation
⚠️ArangoDB arangojsimpl: code/nosql/arangojs.jsy integ--arangodb/integ-arango.test.mjs note: Binary values encode in base64
CouchDB nanoimpl: code/nosql/couchdb.jsy integ--couchdb/integ-couchdb.test.mjs
PouchDB pouchdbimpl: code/nosql/pouchdb.jsy test/nosql/integ_pouchdb.jsy integ--couchdb/integ-pouchdb.test.mjs
Consul KV node-consulimpl: code/nosql/consulkv.jsy integ--consulkv/integ-consul.test.mjs
AWS DynamoDB aws-sdkimpl: code/nosql/dynamodb.jsy integ--dynamodb/integ-dynamodb.test.mjs
Memcache memjsimpl: code/nosql/memjs.jsy integ--memcache/integ-memjs.test.mjs
MongoDB mongojsimpl: code/nosql/mongojs.jsy integ--mongodb/integ-mongodb.test.mjs
FerretDB mongojsimpl: code/nosql/mongojs.jsy integ--mongodb/integ-mongodb.test.mjs
Redis ioredisimpl: code/nosql/ioredis.jsy integ--redis/integ-ioredis.test.mjs
Valkey ioredisimpl: code/nosql/ioredis.jsy integ--redis/integ-ioredis.test.mjs
RethinkDB rethinkdbimpl: code/nosql/rethinkdb.jsy integ--rethinkdb/integ-rethinkdb.test.mjs
S3 NoSQL
Integ TestTechnology / LibraryImplementation
MinIO minioimpl: code/nosql/minio.jsy integ--core/integ-minio.test.mjs
AWS S3 or compatible aws4fetchimpl: code/nosql/s3_aws4fetch.jsy test/unittest.web.mjs integ--core/integ-aws4fetch.test.mjs

SQL and Knex adapter

Integ TestTechnology / LibraryImplementation
SQLite import {DatabaseSync} from 'node:sqlite'impl: code/sql/better_sqlite3.jsy integ--sqlite/integ-sqlite-nodejs.test.mjs
SQLite better-sqlite3impl: code/sql/better_sqlite3.jsy integ--sqlite/integ-better-sqlite3.test.mjs
SQLite node-sqlite3impl: code/sql/sqlite.jsy integ--sqlite/integ-sqlite3.test.mjs
SQLite knex.jsimpl: code/sql/knex.jsy integ--sqlite3/integ-knex-sqlite.test.mjs
PostgreSQL knex.jsimpl: code/sql/knex.jsy integ--postgres/integ-postgres-knex.test.mjs
CockroachDB knex.jsimpl: code/sql/knex.jsy integ--postgres/integ-cockroach-knex.test.mjs
CrateDB knex.jsimpl: code/sql/knex.jsy note: Crate SQL lacks BLOB in-table support
MariaDB knex.jsimpl: code/sql/knex.jsy integ--maria-mysql/integ-knex-maria-mysql.test.mjs note: Uses MEDIUMBLOB
MySQL knex.jsimpl: code/sql/knex.jsy integ--maria-mysql/integ-knex-maria-mysql.test.mjs note: Uses MEDIUMBLOB
Microsoft SQL Server knex.jsimpl: code/sql/knex.jsy integ--mssql-server/integ-mssql.test.mjs

Keyv adapter

Integ TestTechnology / LibraryImplementation
JavaScript Map keyv and new Map()impl: code/adapter/keyv.jsy integ--local-filesystem/integ-keyv-map.test.jsy
⚠️One file keyv and keyv-fileimpl: code/adapter/keyv.jsy integ--local-filesystem/integ-keyv-file.test.jsy Stored in a single JSON file with binary values encode in base64
Memcache keyv and keyv-memcacheimpl: code/adapter/keyv.jsy integ--memcache/integ-keyv-memjs.test.mjs
MongoDB keyv and @keyv/mongoimpl: code/adapter/keyv.jsy integ--mongodb/integ-keyv-mongodb.test.mjs
SQLite keyv and @keyv/sqliteimpl: code/adapter/keyv.jsy integ--sqlite3/integ-keyv-sqlite.test.mjs

About PHORBAS

The PHORBAS project implements a persistent binary data storage scheme that accomodates opaque (encrypted) information. It is inspired by binary content addressable data storage, replication features of CouchDB, the opaque data storage of Tahoe-LAFS, the eventual consistency of CRDTs, and persistent Hash array mapped trie data structures.

Phorbas appears in in Greek mythology and is connected to "giving pasture" -- in this case, safe pasture to for data storage.

P - Persistent
H - Hashed
O - Opaque
R - Replicable
B - Binary content
A - Addressable
S - Store

Due to the opaque nature of the data storage, the storage is not queryable like a database. It is similar to IPFS with a less ambitious goal of being a pratical data storage scheme for data for embedding into other projects.

License

2-Clause BSD

0.5.0

8 months ago

0.5.1

8 months ago

0.4.3

3 years ago

0.4.1

4 years ago

0.4.0

4 years ago

0.4.2

4 years ago

0.3.2

4 years ago

0.3.4

4 years ago

0.3.3

4 years ago

0.3.1

4 years ago

0.2.15

4 years ago

0.3.0

4 years ago

0.2.14

4 years ago

0.2.13

4 years ago

0.2.12

4 years ago

0.2.11

4 years ago

0.2.10

4 years ago

0.2.7

4 years ago

0.2.9

4 years ago

0.2.8

4 years ago

0.2.6

4 years ago

0.2.1

4 years ago

0.2.3

4 years ago

0.2.2

4 years ago

0.2.5

4 years ago

0.2.4

4 years ago

0.2.0

4 years ago

0.1.5

5 years ago

0.1.4

5 years ago

0.1.3

5 years ago

0.1.2

6 years ago

0.1.1

6 years ago

0.1.0

6 years ago

0.0.1

6 years ago