bireader

BiReader / BiWriter

A fast, dual-mode (sync / async) file / buffer handler with byte + bit-level access.

Feature rich binary reader and writer that keeps track of your position to quickly create file structures. Perfect for binary parsers, editors, game save files, custom formats, or any situation where you need random access + structural modifications without loading the entire file into memory. Includes shared naming conventions, programmable inputs and advanced math for easy data conversions on low level parsing. Accepts Uint8Array, Buffer or a filePath. Includes Sync and Async versions.

Features

• Dual mode: Sync or Async file reader (r+ / r) on disk or pure in-memory Buffer or Uint8Array
• Chunked async loading – configurable windowSize (default 4 KiB)
  → Set windowSize: 0 to load the entire file in one async read
• Byte cursor: Track and change location with offset + bit cursor bitOffset
• Full bitfield support – readBit() / writeBit() with:
  • signed / unsigned
  • big-endian (be) or little-endian (le)
  • any alignment (bits can start anywhere)
• Structural edits:
  • insert() – insert data anywhere (grows the buffer)
  • delete() – remove data and return the removed chunk
  • trim() / clip() – shrink from the cursor to the end (returns removed tail)
  • push() / append() – add to the end; unshift() / prepend() – add to the start
  • replace() / fill() / extract() – overwrite, fill-a-range, or copy-out
• Expandable files with smart growthIncrement (default 1 MiB) to minimize syscalls
• Concurrency-safe async – every async op is serialized on its instance, plus cursor-free *At(offset) reads/writes and runExclusive() for atomic sequences (see Async)
• get() / return() – flushes changes and returns the current content
• readOnly and strict modes (for limiting growthIncrement)
• Built on a small, fully-tested engine (v5) - DataView-based codecs, strict-null-safe
• Zero dependencies (only fs & fs/promises in Node)

Supported data types

• Bitfields ([u]bit{1-32}{le|be}) 1-32 bit signed or unsigned value in big or little endian order
• Bytes ([u]int8, byte) 8 bit signed or unsigned value
• Shorts ([u]int16, word, short{le|be}) 16 bit signed or unsigned value in big or little endian order
• Half Floats (halffloat, half{le|be}) 16 bit decimal value in big or little endian order
• Integers ([u]int32, long, int, dword{le|be}) 32 bit signed or unsigned value in big or little endian order
• Floats (float{le|be}) 32 bit decimal value in big or little endian
• Quadwords ([u]int64, quad, bigint{le|be}) 64 bit signed or unsigned in big or little endian
• Double Floats (doublefloat, dfloat{le|be}) 64 bit decimal value in big or little endian
• Strings (string) Fixed and non-fixed length, UTF, pascal, wide pascal. Includes all TextEncoder types

What's New?

v5

• Internals rewritten into a small, unit-tested engine - the two legacy base classes were replaced and deleted (~9,600 lines), the facades shrank ~57%, and the mechanical aliases (uint32le, bit8be, …) are now generated from a single table.
• Concurrency-safe async: every async operation on an instance is serialized by a built-in queue (overlapping cursor calls no longer corrupt state). Added cursor-free *At(offset) reads/writes (safe to call concurrently) and runExclusive() for atomic sequences.
• Faster async strings (single batched read/write) and a lighter close() in file mode.
• Whole codebase compiles under strictNullChecks; the pre-DataView fallback codecs were removed (every supported runtime has DataView; only the manual float16 path remains).

v4

• Added BiReaderAsync and BiWriterAsync. See Async classes.
• Uses DataView read and write functions when possible for more efficient code (previous code is now fallback).
• Added support for UTF-32 and Double Wide Pascal (32 bit) strings.
• Large code clean up with included test.
• Marked deprecated BiReaderStream and BiWriterStream as functionality was moved to BiReader and BiWriter for file reading (Node only).
• Values for writes are now clamped to bit size and don't throw errors.

v3

• Added enforceBigInt option for always returning a BigInt type on 64 bit reads, otherwise will return a number if integer safe.
• Added Browser, Node CommonJS and Node ESM modules.
• Added new BiReaderStream and BiWriterStream (Node only).
• Added .deleteFile() and .renameFile(filePath).
• Added setter .strSettings for use with .str for easier coding.
• Added better options for extending array buffer when writing data with growthIncrement.
• Consolidated all options argument into single object when creating class.
• Removed deprecated bireader and biwriter classes.
• Fixed standalone hexdump function.

v2

• Created new BiReader and BiWriter classes with get and set functions for easier coding.
• Marked bireader and biwriter as deprecated. Set to be removed next update.

v1

• Included math functions and value searches.
• Many bug fixes.

Installation

npm install bireader

Provides both CommonJS and ES modules. Works in Browser, Node.js (CJS + ESM), and provides zero-dependency binaries.

Quick Start – The 4 Classes

1. BiReader + BiWriter (Sync – Recommended for 99% of use cases)

import { BiReader, BiWriter } from 'bireader';

// === Reading from Buffer / Uint8Array ===
const data = new Uint8Array([0x01, 0x02, 0x03, 0x04 /* ... */]);
const br = new BiReader(data);

console.log(br.uint32le);              // auto-advances cursor
console.log(br.halffloatle);
console.log(br.string({ length: 10 })); // or presets: .cstring(), .pstring2le(), .utf16string()

// === Writing (auto-grows) ===
const bw = new BiWriter();
bw.uint32le = 0xCAFEBABE;
bw.halffloatle = 3.1416;
bw.pstring2le("Hello World");
bw.writeBit(0b101, 3);                 // bit-level control

const finalBuffer = bw.data;           // the current buffer (or bw.get())

Node.js file support (still sync)

const brFile = new BiReader('huge-but-not-gigantic.bin'); // accepts filePath
console.log(brFile.int64le);

2. BiReaderAsync + BiWriterAsync (Async – for huge files)

import { BiReaderAsync, BiWriterAsync } from 'bireader';

// === Async Reader (random access, no full load into RAM) ===
const brAsync = await BiReaderAsync.create('massive-50gb-file.bin');

await brAsync.goto(1024 * 1024 * 1024);    // jump to the 1 GB mark (absolute)
const magic = await brAsync.str();         // in async classes get/set become methods: await brAsync.uint32le(), etc.
const value = await brAsync.readUInt64();  // all methods are now async

await brAsync.close();

Async Writer

const bwAsync = await BiWriterAsync.create('output-huge.bin');

await bwAsync.writeUInt32(0xDEADBEEF);
await bwAsync.halffloatle(1.618);
await bwAsync.writeString("Header data", { stringType: 'utf-8', terminateValue: 0 });

await bwAsync.close();   // flushes everything

Important: BiReaderAsync / BiWriterAsync are Node.js only (they use fs/promises).

In the async classes, the get/set presets from the sync classes become methods: br.uint32le → await br.uint32le(), and bw.uint32le = x → await bw.uint32le(x).

3. Bit-Field Example (works on all 4 classes)

// Reading (BiReader)
const br = new BiReader(myData);
br.goto(0x100);

// Bit-level presets (auto-advance cursor)
console.log(br.ubit4);     // 4 bits, unsigned
console.log(br.bit8);      // signed 8 bits
console.log(br.ubit24be);  // 24 bits big-endian

br.insetBit = 3;           // manual bit control within the current byte
console.log(br.readBit(5));        // read any number of bits (1-32)

// Writing (BiWriter - a BiReader is read-only by default)
const bw = new BiWriter(new Uint8Array(16));
bw.writeBit(0b10110, 5);           // write any number of bits (1-32)
bw.ubit4 = 0xF;                    // preset setters also work

4. String Handling (all variants)

const bw = new BiWriter();
bw.strSettings = { length: 8, stringType: 'utf-16', terminateValue: 0 };

bw.str = "Hello 🌍";              // uses current strSettings
bw.pstring2le("Pascal string");
bw.utf16string("UTF-16 wide string"); // aka .unistring()

const br = new BiReader(bw.data);
console.log(br.cstring());          // null-terminated
console.log(br.pstring4be());

5. Math Helpers (XOR, shifts, etc.)

const bw = new BiWriter(data);
bw.xor(0xAA);                  // XOR entire buffer with key
bw.lShift(1, 0, 8);            // left-shift first 8 bytes by 1 bit
bw.and(0x0F, 0, 4);            // AND bytes 0-3 with 0x0F
bw.hexdump();                  // console.log a hex dump (or { returnString: true })

6. Concurrency (async only - v5)

The async classes share one cursor, so overlapping cursor-based calls on the same instance are serialized automatically. For explicit control:

const r = await BiReaderAsync.create('data.bin');

// runExclusive: run a sequence atomically (queued if another op is in flight)
const [a, b] = await Promise.all([
  r.runExclusive(() => r.readUInt32()),
  r.runExclusive(() => r.readUInt32()),
]);                                   // deterministic, sequential

// *At(offset): cursor-free reads/writes - safe to call concurrently, never move the cursor
const [header, len] = await Promise.all([
  r.readUInt32At(0, 'big'),
  r.readUInt16At(8, 'little'),
]);
// write side: writeUInt32At, writeBytesAt, writeFloat64At, writeBigInt64At, ...

7. Real-World Complete Example (WebP parser)

import {BiReader, BiWriter} from 'bireader';

// read example - parse a webp file
function parseWebp(data){
  const br = new BiReader(data);
  br.strSettings = {length: 4};
  br.hexdump({suppressUnicode:true}); // console.log data as hex

  //         0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F  0123456789ABCDEF
  // 00000  52 49 46 46 98 3a 00 00 57 45 42 50 56 50 38 58  RIFF.:..WEBPVP8X
  // 00010  0a 00 00 00 10 00 00 00 ff 00 00 ff 00 00 41 4c  ..............AL
  // 00020  50 48 26 10 00 00 01 19 45 6d 1b 49 4a 3b cf 0c  PH&.....Em.IJ;..
  // 00030  7f c0 7b 60 88 e8 ff 04 80 a2 82 65 56 d2 d2 86  ..{`.......eV...
  // 00040  24 54 61 d0 83 8f 7f 0e 82 b6 6d e3 f0 a7 bd ed  $Ta.......m.....
  // 00050  87 10 11 13 40 3b 86 8f 26 4b d6 2a b7 6d 24 39  ....@;..&K.*.m$9
  // 00060  52 4f fe 39 7f 3b 62 4e cc ec 9b 17 31 01 0c 24  RO.9.;bN....1..$
  // 00070  49 89 23 e0 01 ab 52 64 e3 23 fc 61 db 76 cc 91  I.#...Rd.#.a.v..
  // 00080  b6 7d fb 51 48 c5 69 db 4c 1b 63 db b6 ed b9 6d  .}.QH.i.L.c....m
  // 00090  db be 87 8d b1 6d db 9e b6 cd a4 d3 ee 24 95 54  .....m.......$.T
  // 000a0  52 b8 8e 65 a9 eb 38 ce ab 52 75 9d 67 ff 75 2f  R..e..8..Ru.g.u/
  // 000b0  77 44 40 94 6d 25 6c 74 91 a8 88 86 58 9b da 6e  wD@.m%lt....X..n

  const header = {};
  header.magic = br.str;                // RIFF
  header.size = br.uint32le;            // 15000
  header.fileSize = header.size + 8;    // 15008
  header.payload = br.str;              // WEBP
  header.format = br.str;               // VP8X
  header.formatChunkSize = br.uint32le; // 10
  switch (header.format){
    case "VP8 ":
        header.formatType = "Lossy";
        var readSize = 0;
        header.frame_tag = br.ubit24;
        readSize += 3;
        header.key_frame = header.frame_tag & 0x1;
        header.version = (header.frame_tag >> 1) & 0x7;
        header.show_frame = (header.frame_tag >> 4) & 0x1;
        header.first_part_size = (header.frame_tag >> 5) & 0x7FFFF;
        header.start_code = br.ubit24; // should be 2752925
        header.horizontal_size_code = br.ubit16;
        header.width = header.horizontal_size_code & 0x3FFF;
        header.horizontal_scale = header.horizontal_size_code >> 14;
        header.vertical_size_code = br.ubit16;
        header.height = header.vertical_size_code & 0x3FFF;
        header.vertical_scale = header.vertical_size_code >> 14;
        readSize += 7;
        header.VP8data = br.extract(header.formatChunkSize - readSize, true);
        break;
    case "VP8L":
        header.formatType = "Lossless";
        var readSize = 0;
        header.signature = br.ubyte; // should be 47
        readSize += 1;
        header.readWidth =  br.ubit14;
        header.width = header.readWidth+1;
        header.readHeight =  br.ubit14;
        header.height = header.readHeight+1;
        header.alpha_is_used =  br.bit1;
        header.version_number =  br.ubit3;
        readSize += 4;
        header.VP8Ldata = br.extract(header.formatChunkSize - readSize, true);
        break;
    case "VP8X":
        header.formatType = "Extended";
        br.big();              // switch to Big Endian bit read
        header.rsv = br.bit2;  // Reserved
        header.I = br.bit1;    // ICC profile
        header.L = br.bit1;    // Alpha
        header.E = br.bit1;    // Exif
        header.X = br.bit1;    // XMP
        header.A = br.bit1;    // Animation
        header.R = br.bit1;    // Reserved
        br.little();           // return to little
        header.rsv2 = br.ubit24;
        header.widthMinus1 = br.ubit24;
        header.width = header.widthMinus1 + 1
        header.heightMinus1 = br.ubit24;
        header.height = header.heightMinus1 + 1
        if(header.I)
        {
          header.ICCP = br.str;  // Should be ICCP
          header.ICCPChunkSize = br.uint32;
          header.ICCPData = br.extract(header.ICCPChunkSize, true);
        }
        if(header.L)
        {
          header.ALPH = br.str;  // Should be ALPH
          header.ALPHChunkSize = br.uint32;     // 4134
          header.ALPHData = br.extract(header.ALPHChunkSize, true);
        }
        if(header.A)
        {
          header.ANI = br.str;  // Should be ANIM or ANIF
          header.ANIChunkSize = br.uint32;
          if(header.ANI == "ANIM")
          {
            header.BGColor = br.uint32;
            header.loopCount = br.ushort;
            header.ANIMData = br.extract(header.ANIChunkSize, true);
          } else
          if (header.ANI == "ANIF")
          {
            header.FrameX = br.ubit24;
            header.FrameY = br.ubit24;
            header.readFrameWidth = br.ubit24;
            header.readFrameHeight = br.ubit24;
            header.frameWidth = readFrameWidth + 1;
            header.frameHeight = readFrameHeight + 1;
            header.duration = br.ubit24;
            header.rsv3 = br.ubit6;
            header.byte.B = br.bit1; // Blending
            header.byte.D = br.bit1; // Disposal
            header.frameData = br.extract(16, true);
            header.ANIFData = br.extract(header.ANIChunkSize, true);
          }
        }
        header.extFormatStr = br.str;
        header.extChunkSize = br.uint32;
        header.extData = br.extract(header.extChunkSize, true);
        if(header.E)
        {
          header.EXIF = br.str;  // Should be EXIF
          header.EXIFChunkSize = br.uint32;
          header.EXIFData = br.extract(header.EXIFChunkSize, true);
        }
        if(header.X)
        {
          header.XMP = br.str;  // Should be XMP
          header.XMPChunkSize = br.uint32;
          header.XMPMetaData = br.extract(header.XMPChunkSize, true);
        }
        break;
    default:
        header.data = br.extract(header.formatChunkSize, true);
        break;
  }
  br.finished();
  return header;
}

// write example - write a webp file from read data
function write_webp(data){
  const bw = new BiWriter(new Uint8Arry(0x100000)); // Will extends array as we 
  // write if needed by default
  bw.strSettings = {length: 4};
  bw.str = "RIFF";
  bw.uint32le = 0; // dummy for now, will be final size - 8
  bw.str = "WEBP";
  switch(data.format){
    case "VP8 ":
      bw.str = "VP8 ";
      bw.uint32le = data.VP8data.length;
      bw.ubit24 = data.key_frame;
      bw.ubit24 = data.start_code;
      bw.ubit16 = data.horizontal_size_code;
      bw.ubit16 = data.vertical_size_code;
      bw.overwrite(data.VP8data ,true);
      break;
    case "VP8L":
      bw.str = "VP8L";
      bw.uint32le = data.VP8Ldata.length - 4;
      bw.ubyte = 47;
      bw.ubit14 = data.width - 1;
      bw.ubit14 = data.heigth - 1;
      bw.ubit1 = data.alpha_is_used;
      bw.bit3 = data.version_number;
      bw.overwrite(data.VP8Ldata,true);
      break;
    case "VP8X":
      bw.str = "VP8X";
      bw.uint32le = 10;
      bw.big();
      bw.bit2 = 0;
      bw.bit1 = data.I;
      bw.bit1 = data.L;
      bw.bit1 = data.E;
      bw.bit1 = data.X;
      bw.bit1 = data.A;
      bw.bit1 = 0;
      bw.little();
      bw.ubit24 = data.rsv2;
      bw.ubit24 = data.width - 1;
      bw.ubit24 = data.height - 1;
      if(data.I)
      {
        bw.str = data.ICCP;
        bw.uint32 = data.ICCPData.length;;
        bw.replace(data.ICCPData, true);
      }
      if(data.L)
      {
        bw.str = data.ALPH;
        bw.uint32 = data.ALPHData.length;
        bw.replace(data.ALPHData);
      }
      if(data.A)
      {
        bw.str = data.ANI;
        bw.uint32 = data.ANIChunkSize;
        if(data.ANI == "ANIM")
        {
          bw.uint32 = data.BGColor;
          bw.ushort = data.loopCount;
          bw.replace(data.ANIMData);
        } else
        if (data.ANI == "ANIF")
        {
          bw.ubit24 = data.FrameX;
          bw.ubit24 = data.FrameY;
          bw.ubit24 = data.frameWidth - 1;
          bw.ubit24 = data.frameHeigh - 1;
          bw.ubit24 = data.duration;
          bw.ubit6  data.rsv3;
          bw.bit1 = data.byte.B;
          bw.bit1 = data.byte.D;
          bw.replace(data.frameData, true);
          bw.replace(data.ANIFData, true);
        }
      }
      bw.str = data.extFormatStr;
      bw.uint32 = data.extData.length;
      bw.replace(data.extData, true);
      if(data.E)
      {
        bw.str = data.EXIF;
        bw.uint32 = data.EXIFData.length;
        bw.replace( data.EXIFData, true);
      }
      if(data.X)
      {
        bw.str = data.XMP;
        bw.uint32 = data.XMPMetaData.length;
        bw.replace(data.XMPMetaData, true);
      }
      break;
    default:
      break;
  }
  bw.trim(); // remove any remaining bytes
  bw.goto(4);
  bw.uint32le = bw.size - 8; // write file size
  return bw.return();
}

Common Functions

Common functions for setup, movement, manipulation and math shared by both.

Naming is shared across sync and async classes.

Async

With 4.0 you can now use BiReaderAsync and BiWriterAsync for async operations. Pass a normal Buffer, Uint8Array or a string path (only in Node.js). When passed a Buffer or Uint8Array, it uses the same logic as the sync class. When reading or creating a file, the class loads the file in chunks for quick editing (the chunk size is configurable via windowSize). This class is designed for larger files where you don't want to load the whole file buffer into memory all at once, or when you need an async class.

Naming: Same function naming applies to async as the Common Functions section, but these classes use all async functions, so the get / set presets from the sync classes become async methods (br.uint32le → await br.uint32le(), bw.uint32le = x → await bw.uint32le(x)).

Async-only additions (v5):

• runExclusive(fn) - run a sequence of cursor operations atomically; overlapping calls on one instance are queued and run one at a time (reentrant).
• readXAt(offset, …) / writeXAt(offset, …) - read/write at an absolute offset without moving the cursor, so they're safe to call concurrently. Available for UInt8, Int16/UInt16, Int32/UInt32, Float32, Float64, BigInt64/BigUInt64, and raw Bytes.

See Concurrency for examples.

Bit field

Parse value as a bit field. There are 32 functions from bit1 to bit32 and can be signed or unsigned (with a u at the start) and in little or big endian order (be or le at the end).

Note: Remaining bits are dropped when returning to a byte read. Example, after using bit4 then ubyte, the read location drops the remaining 4 bits after bit4 when reading ubyte. The bitN presets are signed (the top bit is the sign) and the ubitN presets are unsigned; a 1-bit value is always unsigned.

Byte

Parse value as a byte (aka int8). Can be signed or unsigned (with a u at the start).

Short

Parse value as a int16 (aka short or word). Can be signed or unsigned (with a u at the start) and in little or big endian order (be or le at the end).

Half Float

Parse value as a half float (aka half). Can be in little or big endian order (be or le at the end).

Integer

Parse value as a int32 (aka int, long or dword). Can be signed or unsigned (with a u at the start) and in little or big endian order (be or le at the end).

v5 breaking change: the 32-bit double alias was renamed to dword (it's a 32-bit "double word", not an 8-byte float - that's doublefloat / dfloat).

Float

Parse value as a float. Can be in little or big endian order (be or le at the end).

Quadword

Parse value as a int64 (aka quad or bigint). Can be signed or unsigned (with a u at the start) and in little or big endian order (be or le at the end).

Double Float

Parse value as a double float (aka dfloat). Can be in little or big endian order (be or le at the end).

Strings

Parse a string in any format. Either null terminated strings (utf) or fixed length (pascal). Be sure to use options object for formatting unless using a preset. Default string settings can be stored in strSettings. Strings with larger than 1 byte character reads can use be or le at the end for little or big endian.

Presents include C or Unicode, Ansi and multiple pascals.

Acknowledgements

This project was born from the desire to have a single library that could both read and write in binary with common named functions. Having been using tools like Binary-parser, QuickBMS and 010 Editor in the past, I wanted something I could translate quickly to a Node app and then use in a web site without having to redo work.

I'm happy to connect and grow this library if others find it useful. Pull requests or bug reports are welcome!

License

MIT