@entelligentsia/forge-compress

@entelligentsia/forge-compress

Token-efficient compression for Forge SDLC tool outputs — store queries, entities, validation reports, and arbitrary markdown.

Forge agents read store queries, entity JSON, and markdown artifacts on every turn. Verbose outputs burn tokens fast. This library compresses those outputs into a form that preserves load-bearing facts (IDs, statuses, relationships, key changes) while dropping boilerplate, deep nesting, and repeated structure.

Install

npm install @entelligentsia/forge-compress

Requires Node ≥ 18. Ships ESM only.

Quick start

import {
  compressStoreQuery,
  compressEntity,
  compressMarkdown,
  compressValidateStore,
} from "@entelligentsia/forge-compress";

// Forge store query envelope → compact text
const compact = compressStoreQuery(rawJson, { maxTokens: 2000 });

// Task / bug / feature entity JSON → flat summary
const summary = compressEntity(entityJson, { keepSummaries: "latest" });

// Long markdown artifact → headings + key lines under a budget
const map = compressMarkdown(doc, { mode: "map", maxTokens: 800 });

// validate-store report → grouped errors/warnings, top-N per category
const report = compressValidateStore(reportJson);

What's included

Forge-aware compressors (src/forge/)

Function	Input	Strategy
compressStoreQuery	Forge query envelope JSON	Drop traversal trace and config, keep IDs/titles/statuses/relationships, truncate excerpts
compressEntity / compressEntityList	Entity record JSON (task/bug/feature/sprint)	Flatten phase summaries, keep latest verdicts and key changes
compressMarkdown	Any markdown	Three modes: map (headings + signal lines), headings (TOC only), truncate (head-of-budget)
compressValidateStore	validate-store JSON report	Group by category, show top-N per group, summarize the rest

Generic primitives (src/)

Function	Purpose
countTokens(text)	Heuristic token count (chars ÷ 3.7)
truncateToTokenBudget(text, n)	Cut to a token budget on a newline boundary
lightweightCleanup(text)	Collapse brace-only runs, strip blank-line repeats
verbatimCompact(text)	Lossless whitespace and boilerplate strip
stripAnsi(text)	Remove ANSI escape sequences
compressIb(text)	Information-bottleneck compression driven by per-line entropy
compressProgressive(items, budget)	Tiered budget allocation across an ordered list (recent items get exponentially more tokens)

Design notes

• Lossy by intent. Compressors drop fields agents have proven they don't need (timestamps, hashes, deep config blobs). When you need full fidelity, read the source JSON directly.
• Token-bounded, not byte-bounded. Every public API accepts maxTokens and trims on token boundaries, not bytes.
• No model dependency. Pure functions. No network, no LLM calls — safe to run inside tool handlers.
• Stable shapes. Output is deterministic for a given input and option set, so it caches well in prompt-cache prefixes.

API reference

Types are exported alongside each function — see src/index.ts for the full surface. All compressors return string; pass JSON in as a raw string (they parse internally and fall back to the input on parse error).

Development

npm install
npm run build      # tsc → dist/
npm test           # vitest
npm run lint       # biome

Tests live in src/__tests__/.

License

MIT. See LICENSE.