@devmor-j/readline-pager-linux-musl-arm64

readline-pager

High-performance paginated file reader for Node.js. Process large text files efficiently without loading them into memory.

• Zero dependencies
• Up to ~3× faster than Node.js readline
• Up to ~6× faster with optional native C++ acceleration
• Async (for await...of) and sync (for...of) iteration
• Page-based reading with manual control (next, nextSync)
• Forward and backward reading support
• Fully typed with high test coverage

Important:
Performance depends heavily on the chunkSize option. Tune it for your storage device. A value of 64 KiB is usually a good starting point. Increasing it may improve throughput until you reach the best value for your hardware.

---

Installation

npm install readline-pager

---

Quick start

import { createPager } from "readline-pager";

for await (const page of createPager("./bigfile.txt")) {
  console.log(page[0]);
}

---

Other usage patterns

import { createPager, createNativePager } from "readline-pager";

// Sync iteration
for (const page of createPager("./bigfile.txt")) {
}

// Manual async
const pager = createPager("./bigfile.txt");
while (true) {
  const page = await pager.next();
  if (!page) break;
}

// Manual sync
let page;
const pager = createPager("./bigfile.txt");
while ((page = pager.nextSync()) !== null) {}

// Native C++
for (const page of createNativePager("./bigfile.txt")) {
}

// Automatic cleanup on exit (optional, requires Node.js v22+)
await using pager = createPager("./bigfile.txt");
for await (const page of pager) {
}
// pager will be disposed automatically when it goes out of scope

---

## ⚙️ Options

```ts
createPager(filepath, {
  chunkSize?: number,           // default: 64 * 1024 (64 KiB)
  pageSize?: number,            // default: 1_000
  delimiter?: string,           // default: "\n"
  prefetch?: number,            // default: 8
  backward?: boolean,           // default: false
  output?: "string" | "buffer", // default: "string"
});

createNativePager(filepath, {
  pageSize?: number,            // default: 1_000
  delimiter?: string,           // default: "\n"
  backward?: boolean,           // default: false
  output?: "string" | "buffer", // default: "string"
});

• chunkSize — Number of bytes read per I/O operation.
• pageSize — Number of lines per page.
• delimiter — Line separator.
• prefetch — Maximum number of pages buffered internally.
• backward — Read the file from end to start.
• output — Controls the page data type.

Note: createNativePager requires x86 AVX2 or ARM NEON CPU instruction set extensions and will throw if they are not available. It also does not support multi-character delimiters because it uses fast SIMD-based scanning.

---

API

pager.next(): Promise<string[] | Buffer | null>

Returns the next page asynchronously.

Returns null when the end of the file is reached.

Empty lines are preserved.

Note: Unlike Node.js readline, which may skip empty files or leading empty lines, readline-pager always returns all lines.

• A completely empty file (0 bytes) produces [""] on the first read.
• A file containing multiple empty lines returns each line as an empty string.

pager.nextSync(): string[] | Buffer | null

Synchronous version of pager.next().

Returns the next page immediately or null when the end of the file is reached.

pager.close(): Promise<void>

Stops reading and releases resources asynchronously. Safe to call at any time.

---

Benchmark

Run the benchmark locally:

npm run benchmark                   # default: Node.js
npm run benchmark -- deno           # Deno runtime
npm run benchmark -- bun --backward # Bun runtime with custom args

Test setup: generated text files (UUID lines), NVMe SSD, Node.js runtime. Results are averaged across multiple runs. Actual performance depends on hardware.

Throughput (MB/s)

Method	1M lines (35 MB)	10M lines (353 MB)	100M lines (3.5 GB)	1B lines (35.3 GB)
readline	~370 MB/s	~460 MB/s	~460 MB/s	~460 MB/s
readline-pager (JS)	~1100 MB/s	~1300 MB/s	~1300 MB/s	~1150 MB/s
readline-pager (C++)	~2200 MB/s	~2500 MB/s	~2500 MB/s	~2450 MB/s

---

Development & Contributing

• Minimum supported Node.js: v18.12
• Development/test environment: Node v26.3 and TypeScript v6.0

Run tests:

npm i
npm test

Contributions are welcome. Open an issue or submit a PR.

---

License

MIT — Morteza Jamshidi