# @muze-nl/od-jsontag

> On Demand JSONTag: parse/serialize large datastructures on demand, useful for sharing data between threads

Latest version **0.5.0** (published 2026-09-24) · MIT license · 0 weekly downloads

## Install

```sh
npm install @muze-nl/od-jsontag
pnpm add @muze-nl/od-jsontag
yarn add @muze-nl/od-jsontag
bun add @muze-nl/od-jsontag
```

## Health

**Score 60/100 (C)** — status: active.

Positive: esm support; no vulnerabilities; recently updated; high maintenance score.

Warnings: low downloads; no types; pre 1.0.

## Facts

| | |
|---|---|
| Version | 0.5.0 |
| Published | 2026-09-24 |
| First published | 2024-03-19 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | ESM + CommonJS |
| Node | >=20.0.0 |
| Dependencies | 1 |
| Unpacked size | 108.9 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 0 |
| Author | Auke van Slooten |
| Maintainers | poef |
| Keywords | JSONTag, On Demand Parsing |

## Links

- npm: https://www.npmjs.com/package/@muze-nl/od-jsontag
- Repository: https://github.com/muze-nl/od-jsontag
- Issues: https://github.com/muze-nl/od-jsontag/issues
- npm.io page: https://npm.io/package/@muze-nl/od-jsontag

## Dependencies (1)

- [@muze-nl/jsontag](https://npm.io/package/@muze-nl/jsontag.md) ^0.10.4

## Alternatives

- [@mapbox/jsonlint-lines-primitives](https://npm.io/package/@mapbox/jsonlint-lines-primitives.md) — 5.3M weekly downloads
- [reftools](https://npm.io/package/reftools.md) — 3.5M weekly downloads
- [@hey-api/openapi-ts](https://npm.io/package/@hey-api/openapi-ts.md) — 3.5M weekly downloads
- [@mapbox/geojson-rewind](https://npm.io/package/@mapbox/geojson-rewind.md) — 2.4M weekly downloads
- [turbo-stream](https://npm.io/package/turbo-stream.md) — 1.7M weekly downloads

## Recent versions

- 0.5.0 (latest) — 2026-09-24
- 0.4.6 — 2026-04-16
- 0.4.5 — 2026-01-07
- 0.4.4 — 2026-01-07
- 0.4.3 — 2025-12-17
- 0.4.2 — 2025-11-18
- 0.4.1 — 2025-10-05
- 0.4.0 — 2025-10-04
- 0.3.4 — 2025-09-24
- 0.3.3 — 2024-11-14
- 0.3.2 — 2024-10-31
- 0.3.1 — 2024-10-31
- 0.3.0 — 2024-10-30
- 0.2.9 — 2024-09-23
- 0.2.8 — 2024-08-08
- … 16 more at https://npm.io/package/@muze-nl/od-jsontag/versions

## README

# od-jsontag: On Demand JSONTag

`od-jsontag` is a lazy parser and serializer for large JSONTag-style object graphs.
It stores data as one object per line, keeps references between objects as line
numbers, and creates JavaScript `Proxy` objects so values are parsed only when
your code actually touches them.

The goal is to let you work with large, connected data structures through a
normal object API without paying the full parse cost up front.

```js
import Parser from '@muze-nl/od-jsontag'
import serialize, {stringify} from '@muze-nl/od-jsontag/src/serialize.mjs'

const data = {
  articles: [
    {title: 'On demand parsing'}
  ]
}

const buffer = serialize(data)
const parser = new Parser()
const root = parser.parse(buffer)

// The root proxy exists immediately. Referenced objects are parsed on access.
console.log(root.articles[0].title)

console.log(stringify(buffer))
```

## Why od-jsontag exists

`od-jsontag` was made for data that is too connected and too large to comfortably
parse as one regular JSON document, but still wants to be used like ordinary
JavaScript objects.

JSON is excellent when you want to load a complete tree. It is less ideal when:

- the file is large;
- most requests only touch a small part of the data;
- many objects reference the same object;
- the data needs to be shared between workers;
- identity matters, so repeated references should point at the same object;
- you want to preserve JSONTag metadata such as types and attributes.

`od-jsontag` approaches this by splitting a graph into lines. Line `0` is the
root. Other lines are objects that can be referenced by `~1`, `~2`, and so on.
The parser can scan the file and create lightweight proxies for each line, or it
can use a supplied line index and skip the scan entirely. With an index, only the
root line is parsed at first; referenced lines become proxies and parse lazily
when accessed.

This is especially useful with `SharedArrayBuffer`, because the same serialized
data can be shared with workers without copying the full object graph into each
worker.

## When to use it

Use `od-jsontag` when you have large, mostly-read object graphs and only a
fraction of the objects are needed for a given operation.

Good fits:

- large JSONTag datasets where references and identity matter;
- read-heavy applications that open a large data file and inspect small parts;
- worker-based Node.js applications sharing data through `SharedArrayBuffer`;
- graph-like data where the same entity appears from many paths;
- data with JSONTag object attributes, typed values, or non-enumerable
  properties;
- applications that need access control hooks around object properties.

## When not to use it

Do not use `od-jsontag` just because a file is JSON-shaped. For small files,
`JSON.parse` or `JSONTag.parse` will usually be simpler and faster.

It is probably not the right fit when:

- you always need the full dataset immediately;
- your data is a plain tree with no shared references or identity concerns;
- you need broad query/filter/aggregate operations over millions of rows;
- you need a stable cross-language binary format;
- you need transactional updates, indexing, and persistence like a database.

For analytical workloads that scan a few fields across many similar objects, a
columnar format or database may be a better match. `od-jsontag` is intentionally
object-oriented: it optimizes lazy object access, not column scans.

## Core ideas

### One object per line

Serialized output is a newline-separated sequence of length-prefixed JSONTag
values:

```text
(23){"foo":[~1],"bar":[~2]}
(57)<object class="foo" id="1">{"name":"Foo",#"hidden":"bar"}
(57)<object class="bar" id="2">{"name":"Bar","children":[~1]}
```

Each line starts with `(N)`, where `N` is the byte length of the JSONTag value
after the prefix. References use `~lineNumber`.

See [docs/data-format.md](docs/data-format.md) for a simple explanation of the
format.

### Lazy proxies

`parse()` returns a proxy for the root value. Referenced objects are represented
by proxies as well. The object body is parsed only when code reads a property,
enumerates keys, checks `in`, defines a property, deletes a property, or performs
another operation that needs the object contents.

```js
const root = parser.parse(buffer)

// Parses root, then the referenced object at line 1.
console.log(root.foo[0].name)
```

### Optional line index

If you already have an index of line number to byte positions, pass it as the
second argument to `parse()`:

```js
const index = [
  [0, 28],
  [29, 98],
  [99, 161]
]

const root = parser.parse(buffer, JSON.stringify(index))
```

The index is an array or an object keyed by record number. Each entry is a
`[start, end]` byte range in the data file, with an exclusive end. Sparse changeset
indexes keep their original record numbers. It can be supplied as:

- an already parsed array or record-number object;
- a JSON string;
- a `Uint8Array` containing JSON.

The default parser has no Node imports. For indexed reads it also accepts a
byte source with `byteLength` and a synchronous `read(start, end)` method that
returns exactly that range as a `Uint8Array`. See [portability](docs/portability.md)
for the source contract and runtime requirements.

For Node file descriptors and index-file paths, use the Node entry point:

```js
import Parser from '@muze-nl/od-jsontag/src/node.mjs'
import {openSync, closeSync} from 'node:fs'

const parser = new Parser()
const dataFd = openSync('data.odjt', 'r')

try {
  const root = parser.parse(dataFd, 'data.odjt.index.json')
  console.log(root.foo[0].name)
}
finally {
  closeSync(dataFd)
}
```

Keep every source descriptor open, and its bytes unchanged, while the parser or
its proxies are in use. The parser does not own or close descriptors. Repeated
`parse(fd, index)` calls overlay records at their existing numbers; untouched
records keep their previous file source. Buffer patches also work on indexed
views. Use separate parsers for independently retained historical snapshots.

Read-only parsers cache up to 256 clean decoded records by default:

```js
parser.cacheSize = 128
console.log(parser.cacheInfo())
parser.clearCache()
```

Evicted records reload when accessed; live object proxies keep their identity.
The bound covers decoded record bodies, not the record index, caller-retained
arrays/values, or edits. Mutable sessions retain touched records until the session
is discarded or their updates are applied. See the [reference](docs/reference.md)
for lifetime and error semantics.

## Mutability

Parsers are immutable by default:

```js
const parser = new Parser()
const root = parser.parse(buffer)

root.name = 'New name' // throws
```

Create a mutable parser by passing `false` as the second constructor argument or
by setting `parser.immutable = false`:

```js
const parser = new Parser(undefined, false)
const root = parser.parse(buffer)

root.name = 'New name'
```

Changed objects are serialized again. Unchanged parsed proxies can copy their
original byte range back into the output.

## Access control

You can install an access hook on `parser.meta.access`:

```js
const parser = new Parser()

parser.meta.access = (object, property, method) => {
  return property === 'name'
}

const root = parser.parse(buffer)

console.log(root.name)  // allowed
console.log(root.secret) // undefined
```

The `method` argument is usually one of:

- `get`
- `set`
- `has`
- `deleteProperty`
- `defineProperty`

Access denial returns `undefined` or `false`, depending on the proxy operation.

## Serialization

Use `serialize(value, options)` to create the od-jsontag byte representation.
It returns a `Uint8Array` backed by a `SharedArrayBuffer` when available, or an
`ArrayBuffer` otherwise. For large output,
`serializeChunks(value, options)` yields framed `Uint8Array` chunks incrementally
without allocating the entire serialized dataset. Both visit the complete record
catalog, including records that have never been accessed.

```js
const buffer = serialize(root)
const text = stringify(buffer)
```

Useful options:

- `meta`: share parser metadata such as `resultArray` and the id index.
- `changes: true`: serialize only changed lines as a patch-style stream.
- `skipLength: true`: internal option used when serializing a single line body.

See [docs/reference.md](docs/reference.md) for the API reference.

For benchmark results and memory tradeoffs compared with standard JSON, see
[docs/performance.md](docs/performance.md).

## JSONTag compatibility

`od-jsontag` builds on [`@muze-nl/jsontag`](https://github.com/muze-nl/jsontag/).
It preserves JSONTag types and attributes for serialized values. It also adds
support for non-enumerable object properties by prefixing the property with `#`
inside the line format:

```text
(57)<object class="foo" id="1">{"name":"Foo",#"hidden":"bar"}
```

## Documentation

- [API reference](docs/reference.md)
- [Data format](docs/data-format.md)
- [Performance tradeoffs](docs/performance.md)

## Development

```sh
npm test
npm run lint
node --expose-gc benchmark/retention.mjs
```

Tests report coverage and fail on assertion failures. Incomplete coverage is
explicitly allowed; coverage percentage is evidence, not a claim of completeness.
The file-backed tests include unread records, sparse overlays, partial reads,
Unicode, cache eviction, reflection policy and command-worker session resets.

See [the hardening evaluation](docs/file-backed-hardening.md) for verification
against SimplyStore and the current operating limits.

---
_Source: https://npm.io/package/@muze-nl/od-jsontag · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._