@fetsorn/csvs-js v0.21.2

#+TITLE: csvs-js #+OPTIONS: toc:nil

Create, read, update and delete records in a csvs database, run powerful search queries.

[https://norcivilianlabs.org/csvs]

• Setup #+begin_src sh

install

npm i @fetsorn/csvs-js

test js functions

yarn test

test wasm functions in the browser with index.html

npx http-server

publish

yarn build npm publish #+end_src

• Getting started #+begin_src js // functions exported by csvs-js require a callback // that describes how to fetch and write files from a csvs database

function fetchDataMetadir(path) { // fetch file from path }

function writeDataMetadir(path, content) { // write content to file at path }

const callback = { fetch: fetchDataMetadir, write: writeDataMetadir }

// get array of results from query var searchParams = new URLSearchParams() searchParams.set('val1', 'foo') const query1 = await queryMetadir(searchParams, callback, true) console.log(query1) // {"val1": "foo", // "val2": "bar", // "UUID": "b29b9ee3ad01efde7d8694cea4a37844c677ad807b61fa90c44409a21710035c"}

// if object has no uuid // editEntry creates a new record in the database // and returns it let entryBaz = {"val1": "foo", "val2": "baz"} let entryNew1 = await editEntry(entryBaz, callback) console.log(entryNew1) // {"val1": "foo", // "val2": "baz", // "UUID": "33d6e141e92852d5be1930458c1713c2bed09f42d545bf95c0f6d6e271b4077a"}

const query2 = await queryMetadir(searchParams, callback, true) console.log(query2) // {"val1": "foo", // "val2": "bar", // "UUID": "b29b9ee3ad01efde7d8694cea4a37844c677ad807b61fa90c44409a21710035c"}, // {"val1": "foo", // "val2": "baz", // "UUID": "33d6e141e92852d5be1930458c1713c2bed09f42d545bf95c0f6d6e271b4077a"}

// if object has a uuid // editEntry updates the record with matching uuid // and returns it let entryBarbaz = {"val1": "foo", "val2": "barbaz", "UUID": "33d6e141e92852d5be1930458c1713c2bed09f42d545bf95c0f6d6e271b4077a"} let entryNew2 = await editEntry(entryBarbaz, callback) console.log(entryNew) // {"val1": "foo", // "val2": "barbaz", // "UUID": "33d6e141e92852d5be1930458c1713c2bed09f42d545bf95c0f6d6e271b4077a"}

const query3 = await queryMetadir(searchParams, callback, true) console.log(query3) // {"val1": "foo", // "val2": "bar", // "UUID": "b29b9ee3ad01efde7d8694cea4a37844c677ad807b61fa90c44409a21710035c"}, // {"val1": "foo", // "val2": "barbaz", // "UUID": "33d6e141e92852d5be1930458c1713c2bed09f42d545bf95c0f6d6e271b4077a"}

// deleteEntry deletes a record with matching uuid let uuid = "b29b9ee3ad01efde7d8694cea4a37844c677ad807b61fa90c44409a21710035c" await deleteEntry(uuid, callback)

const query4 = await queryMetadir(searchParams, callback, true) console.log(query4) // {"val1": "foo", // "val2": "barbaz", // "UUID": "33d6e141e92852d5be1930458c1713c2bed09f42d545bf95c0f6d6e271b4077a"} #+end_src

• More csvs projects [https://github.com/fetsorn/csvs-spec] - format description

[https://github.com/fetsorn/qualia] - Web UI and desktop app

[https://github.com/fetsorn/csvs-sh] - command-line interface

[https://github.com/fetsorn/wasm-grep] - ripgrep compiled to WASM

Getting Started

This documentation describes the library for searching and updating CSVS datasets.

Install from the NPM registry

yarn install @fetsorn/csvs-js

Import the library

import { CSVS } from @fetsorn/csvs-js;

Clone a sample csvs dataset

git clone https://gitlab.com/norcivilian-labs/csvs-template-en

Point the dir variable to the dataset. In NodeJS, that would be the path to a dataset directory.

import fs from 'fs';

const dir = "/path/to/csvs-template-en"

const query = new CSVS({ fs, dir });

And search the records

const records = await query.select(new URLSearchParams());
// [{
//   _: 'datum',
//   UUID: '...',
//   datum: 'value1',
//   actdate: '2001-01-01'
// }]

Next, learn more about csvs integration in the Tutorial and the User Guides.

Tutorial

Let's nodejs query, update, delete

Install from the NPM registry

yarn install @fetsorn/csvs-js

Import the library

import { CSVS } from @fetsorn/csvs-js;

Clone a sample csvs dataset

git clone https://gitlab.com/norcivilian-labs/csvs-template-en

Point the dir variable to the dataset. In NodeJS, that would be the path to a dataset directory.

import fs from 'fs';

const dir = "/path/to/csvs-template-en"

And query the records

const records = await new CSVS({ fs, dir }).select(new URLSearchParams());
// [{
//   _: 'datum',
//   UUID: '...',
//   datum: 'value1',
//   actdate: '2001-01-01'
// }]

To edit a record, change its json and pass it back to csvs

const recordNew = { datum: 'value2', ...records[0] }

await new CSVS({ fs, dir }).update(recordNew);

To add a new record, pass a json object without a UUID to CSVS.update(record). The function will return the new record with a generated UUID.

const recordNew = await new CSVS({ fs, dir }).update({
  _: 'datum',
  datum: 'value2',
  actdate: '2003-03-03'
});

To delete an record, pass it to CSVS.delete(record) and the library will remove record UUID from the dataset.

await new CSVS({ fs, dir }).delete(record);

Learn more about csvs in the User Guides.

CSVS in the Browser

Csvs-lib changes a csvs dataset by passing filepaths to the methods on an FS interface. The simplest way is to import from the fs Node module to interact with local filesystem. The browser is barred from filesystem access, but can emulate it using LightningFS, which stores data in IndexedDB.

See BrowserFS and filer for other examples of emulating a filesystem in the browser.

First, initialize the FS instance. The name 'virtualfs' is used to determine the IndexedDb store name. It is recommended to only create one of these instances for the entire lifetime of the application.

const fs = new LightningFS('virtualfs');

Now, let's populate the filesystem with a dataset by cloning it with isomorphic-git.

const { clone } = await import('isomorphic-git');

const http = await import('isomorphic-git/http/web/index.cjs');

const dir = '/csvs-template-en';

const url = 'https://gitlab.com/norcivilian-labs/csvs-template-en';

await clone({
  fs,
  http,
  dir,
  url
})

At this point there is an IndexedDB store called 'virtualfs' that stores a csvs dataste at '/csvs-template-en'. Let's point the dir variable there.

const dir = "/csvs-template-en"

Now query the dataset

const CSVS = await import('@fetsorn/csvs-js');

const entries = await new CSVS({ fs, dir }).select(new URLSearchParams());
// [{
//   _: 'datum',
//   UUID: '...',
//   datum: 'value1',
//   actdate: '2001-01-01'
// }]