@solsonar/solana-shred-parser

@solsonar/solana-shred-parser

Low-latency Solana shred listener and assembler. Receives raw shreds over UDP and reconstructs VersionedTransaction objects in real time — typically tens to hundreds of milliseconds before the same transactions reach the confirmed RPC ledger.

import { ShredListener, ShredParser } from '@solsonar/solana-shred-parser';

const listener = new ShredListener({ port: 20000 });
const parser   = new ShredParser();

listener.on('shred', (msg) => parser.ingest(msg));
parser.on('tx', ({ slot, tx, firstShredTs }) => {
  const sig = Buffer.from(tx.signatures[0]).toString('hex').slice(0, 16);
  console.log(`slot=${slot} sig=${sig} +${Date.now() - firstShredTs}ms`);
});

listener.start();

What this package does

• Binds a UDP socket on the port you choose
• Sets a large kernel receive buffer (default 128 MB) to survive shred bursts
• Reassembles each slot's data shreds into the contiguous entry stream
• Walks the entry stream and emits each VersionedTransaction as soon as enough contiguous shreds have arrived — not at slot close

Two classes, no dependencies beyond @solana/web3.js (peer):

• ShredListener — UDP socket, emits raw shred datagrams
• ShredParser — stateful assembler, emits VersionedTransactions

What this package does not do

• It does not connect to a validator or shred relay for you. You are responsible for arranging shred traffic to reach your UDP port (typically Jito ShredStream, a self-hosted validator forwarding turbine, or a relay your provider runs).
• It does not interpret transactions. There is no DEX awareness, no instruction decoding, no account filtering. You consume the VersionedTransaction and apply your own logic.
• It does not handle erasure-coded recovery from coding shreds. Coding shreds are filtered out — the parser only operates on data shreds. In practice this is rarely a limitation: most slots ship with full data shred coverage.

If you need transaction-level filtering, instruction decoding, address-lookup-table resolution, or DEX classification, build it on top.

Install

npm install @solsonar/solana-shred-parser @solana/web3.js

@solana/web3.js@^1.91.0 is a peer dependency — install it in your project.

Requires Node 18+ (uses native fetch-era APIs and BigInt).

How shreds work (background)

A Solana validator's TPU produces a stream of entries, each containing zero or more transactions. Entries are batched into slots (one slot ≈ 400 ms). To distribute slot data across the cluster, the leader splits each slot into a sequence of shreds — fixed-size UDP datagrams — and broadcasts them via the turbine tree.

Each shred carries:

• A header identifying its slot, index within the slot, and a LAST_IN_SLOT flag
• A payload (up to ~1 KB) — a fragment of the entry stream, plus a signature trailer

To recover a transaction, you need to:

1. Buffer shreds by (slot, index)
2. Concatenate the contiguous prefix from index 0
3. Strip each shred's signature trailer (last 88 bytes of the payload)
4. Walk the resulting byte stream as a sequence of entries and transactions

This package does steps 1–4 and emits each transaction the moment it becomes whole.

API

new ShredListener(options)

option	type	default	notes
port	number	(required)	UDP port to bind
host	string	'0.0.0.0'	Interface to bind to
recvBufferMb	number	128	Kernel receive buffer size in MB

The kernel may cap recvBufferMb below the requested value. On Linux, raise the ceiling with sysctl -w net.core.rmem_max=<bytes>.

Methods

• start() — bind the socket. Throws if called twice.
• stop() — close the socket. Idempotent.
• stats() — { packets, bytes } snapshot.

Events

• 'listening' — { port, host, recvBufferMb } — socket bound and configured.
• 'shred' — (msg: Buffer) — one raw datagram.
• 'error' — (err: Error).
• 'close' — socket closed.

new ShredParser(options?)

option	type	default	notes
maxSlots	number	80	In-flight slots to keep before evicting the oldest

Methods

• ingest(msg: Buffer) — feed one datagram. Non-data shreds and malformed datagrams are dropped silently.
• shredCount(slot: number) — number of shreds buffered for a given slot.
• stats() — { slotsOk, slotsFail, txTotal, txStreamed }.

Events

• 'slotStart' — { slot, ts } — first shred for a new slot. ts is Date.now() and is the reference for the firstShredTs field on 'tx' events.
• 'tx' — { slot, tx, shredCount, firstShredTs } — a transaction has been deserialized.
• 'slot' — { slot, shredCount, txCount } — the last shred of a slot has arrived.

tx is a VersionedTransaction — its message.compiledInstructions, message.staticAccountKeys, and message.addressTableLookups are populated. To resolve account keys behind ALTs you need to fetch the lookup tables yourself (out of scope for this package).

Constants

import {
  DEFAULT_RECV_BUFFER_MB,    // 128
  PAYLOAD_OFFSET,            // 88
  SHRED_SIGNATURE_TRAILER,   // 88
  DEFAULT_MAX_SLOTS,         // 80
} from '@solsonar/solana-shred-parser';

Performance

On a Linux server colocated with a Jito ShredStream relay, a single Node 20 process running the example pipeline reliably handles full mainnet shred volume:

metric	value
UDP packets / second	50–100k (peak ~150k)
Transactions emitted / second	5–10k
Median shred → tx latency	20–50 ms
95p shred → tx latency	100–200 ms
Memory (RSS, steady state)	~250 MB

Numbers depend heavily on your shred source and network locality.

Examples

See examples/ for runnable scripts:

• basic-listener.js — minimal pipeline
• filter-by-program.js — emit only txs that touch a given program ID
• stats.js — drop-rate and throughput counters

Run with node examples/basic-listener.js (after pointing some shred traffic at port 20000).

Pairs well with

• @solsonar/solana-alt-cache — resolve Address Lookup Tables synchronously. Modern Solana txs reference accounts via ALTs; without resolving them, tx.message.staticAccountKeys only shows half the picture.
• @solsonar/yellowstone-grpc-client — Yellowstone (Geyser) gRPC client with built-in reconnect, heartbeats, and live subscription updates. Use it alongside this package when you need both shred-level latency and confirmed account state.

Troubleshooting

No 'shred' events: verify shred traffic is actually arriving with tcpdump -i any -n udp port 20000. Most shred relays require IP allowlisting on their side.

'shred' events but no 'tx' events: typically you are missing leading shreds (index 0…N). The parser cannot start a slot's entry stream without index 0. This is fine — subsequent slots will start cleanly.

Receive buffer smaller than requested: the OS capped it. On Linux: sudo sysctl -w net.core.rmem_max=$((256*1024*1024)). Re-create the listener afterward.

'tx' events stop after a while: check stats() — if slotsFail keeps climbing it usually means shreds are being dropped at the kernel level (buffer too small for your bursts).

License

MIT