bufferfy

bufferfy

A serialization and deserialization library that space-efficiently packs data into buffers.

• Supports all javascript data types.
• Provides accurate typescript types.
• Serializes to a significantly smaller buffer than message pack and JSON stringify.
• Encodes and decodes structured messages faster than message pack and JSON stringify.
• Encode and decode transforms for streams.

3.0.0 Update

This module is now browser compatible, due to this the following changes have been made:

• When handling bytes, Uint8Arrays are now used instead of node buffers.
• Node style streams have been replaced by WebApi streams.

Install

npm i bufferfy

Usage

import { Codec } from 'bufferfy';

export const ExampleCodec = Codec.Object({
	id: Codec.String("hex", 32),
	relatedIds: Codec.Array(Codec.String("hex", 32)),
	createdAt: Codec.VarInt(),
	updatedAt: Codec.VarInt(),
	deletedAt: Codec.Optional(Codec.VarInt()),
});

type ExampleData = CodecType<typeof ExampleCodec>;

const example: ExampleData = {
   // ... values
}

const buffer = ExampleCodec.encode(example)

const data = ExampleCodec.decode(buffer) // returns ExampleData

// Streams
const encoder = ExampleCodec.Encoder(); // Takes values and outputs buffer chunks

encoder.pipe(stream);
encoder.write(value);
encoder.end();

const decoder = ExampleCodec.Decoder(); // Takes buffer chunks and outputs values

decoder.on("data", (data) => {
	// ... logic
});

stream.pipe(decoder);

API

All codecs provide a standard set of methods.

buffer = AnyCodec.encode(data, target?, offset?)

Returns the data serialized into a buffer. A buffer and offset can be provided, otherwise a new buffer will be created.

data = AnyCodec.decode(source, offset?)

Returns the unserialized data from a buffer. Decoding begins at offset (default 0).

number = AnyCodec.byteLength(data)

Returns the byte length of the data if it were serialized.

boolean = AnyCodec.isValid(data)

Returns true if the codec is able to serialize and unserialize provided data.

Type = CodecType<typeof codec>

Returns the value type of the provided codec.

Types

• Abstract
• Any
• Array
• BitField
• Boolean
• Bytes
• Constant
• Float
• Int
• Object
• Record
• String
• Transform
• Tuple
• UInt
• Union
• VarInt

Utilities

• Merge
• Omit
• Pick

Union Codec Ordering

Codec.Union() tests codecs sequentially. First match wins.

Order specific → general. Put Codec.Any() last (matches everything).

// Correct
Codec.Union([Codec.Constant("active"), Codec.String(), Codec.Any()])

// Wrong - Any() shadows everything
Codec.Union([Codec.Any(), Codec.String()])

Encoding does not validate

encode() does not validate its input — call isValid() first if the value might not conform. For unions this has a specific consequence. On encode, a branch is picked by a shallow type check (typeof, Array.isArray, value instanceof Uint8Array, value === constant); the branch's deep isValid check is skipped whenever no later branch could accept a value that passes that shallow test. Selecting this way is what keeps union encode fast.

For a valid union value the selected branch and the resulting bytes are identical to a fully-validated selection, so the wire format is unchanged. For an invalid value that happens to pass a branch's shallow test, that branch is encoded instead of throwing "Value does not match any codec", producing garbage bytes (garbage in, garbage out). isValid() still returns false for such values.

const codec = Codec.Union([Codec.Object({ a: Codec.UInt(8) }), Codec.Null]);

codec.encode({ a: 5 }); // valid   -> bytes unchanged
codec.encode({});       // invalid -> passes the object type check, encodes [0, 0], does not throw
codec.isValid({});      // false

// Guard first when the value may not conform:
if (codec.isValid(value)) codec.encode(value);

Benchmarks

Values used for benchmarks can be found here. Speed measured with vitest bench on 2026-07-11, median of three runs; run-to-run variance applies.

Size (bytes, smaller is better)

The wire format is deterministic, so bufferfy's sizes are fixed.

Spread of Types

bufferfy.size                   50
msgpack.size                    193
JSON.size                       282

Common Types

bufferfy.size                   1050
msgpack.size                    1706
JSON.size                       1775

Speed (ops/sec, higher is better)

On structured messages, the workload it is built for, bufferfy leads decode and edges out msgpack on encode.

Spread of Types

              bufferfy     msgpack        JSON
encode         395,751     387,482     173,038
decode         343,837     266,447     300,674