# @x12i/countex-pipelines

> Generic pre-aggregated processing counters for records moving through pipelines. Mongo source of truth, pluggable hot layer (in-process memory or Redis).

Latest version **1.0.1** (published 2026-06-20) · MIT license · 0 weekly downloads

## Install

```sh
npm install @x12i/countex-pipelines
pnpm add @x12i/countex-pipelines
yarn add @x12i/countex-pipelines
bun add @x12i/countex-pipelines
```

## Health

**Score 60/100 (C)** — status: active.

Positive: has types; esm support; no vulnerabilities; high quality score.

Warnings: low downloads.

## Facts

| | |
|---|---|
| Version | 1.0.1 |
| Published | 2026-06-20 |
| First published | 2026-06-20 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | bundled |
| Module format | ESM + CommonJS |
| Node | >=18 |
| Dependencies | 1 |
| Unpacked size | 75.2 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| Author | x12i |
| Maintainers | x12i |
| Keywords | counters, pipelines, processing, rollups, mongo, redis, analytics |

## Links

- npm: https://www.npmjs.com/package/@x12i/countex-pipelines
- Repository: https://github.com/x12i/countex-mono-repo
- Homepage: https://github.com/x12i/countex-mono-repo/tree/main/packages/countex-pipelines#readme
- Issues: https://github.com/x12i/countex/countex-pipelines/issues
- npm.io page: https://npm.io/package/@x12i/countex-pipelines

## Dependencies (1)

- [mongodb](https://npm.io/package/mongodb.md) ^7.2.0

## Alternatives

- [angular-pipes](https://npm.io/package/angular-pipes.md) — 5.6K weekly downloads
- [@ng-web-apis/midi](https://npm.io/package/@ng-web-apis/midi.md) — 2.6K weekly downloads
- [happn-3](https://npm.io/package/happn-3.md) — 1.6K weekly downloads
- [@opensip-cli/lang-go](https://npm.io/package/@opensip-cli/lang-go.md) — 1.2K weekly downloads
- [mongoose-typescript](https://npm.io/package/mongoose-typescript.md) — 85 weekly downloads

## Recent versions

- 1.0.1 (latest) — 2026-06-20

## README

# @x12i/countex-pipelines

> Generic pre-aggregated **processing counters** for records moving through pipelines.

Countex-pipelines tracks status transitions (`in_progress` → `done` | `failed`) and serves rollups by **subject**, **pipeline**, and **dimension** — sub-second when the hot layer is warm, with **no source-document scans at query time**.

**Package policy:** Generic only. No host-product types, client names, or embedded scan logic. The host supplies historical replay via `CountexRebuildSource` when needed.

---

## Default deployment: single machine, single process

The **simplest and recommended starting config** needs only **MongoDB** — no Redis required.

```bash
export MONGO_URI="mongodb://127.0.0.1:27017"
# REDIS_URL is optional — omit it for single-machine mode
```

```typescript
import { createCountexFromEnv } from "@x12i/countex-pipelines";

const client = await createCountexFromEnv();
await client.initialize({ tenantId: "tenant-1", fromCounterStore: true });

await client.ingest({
  tenantId: "tenant-1",
  subject: "entities",
  recordId: "rec-42",
  pipelineId: "enrich",
  status: "done",
  occurredAt: new Date().toISOString(),
});

const rollup = await client.rollup({ tenantId: "tenant-1" });
console.log(rollup.bySubject, rollup.byPipeline);

await client.close();
```

When `REDIS_URL` is **absent**, countex-pipelines uses an **in-process `MemoryHotStore`**. Mongo remains the **source of truth**; on restart the hot layer is empty and is **re-warmed from Mongo** automatically on initialize or first query. **Do not run multiple processes with `MemoryHotStore`** — use Redis for multi-server.

| Config | Hot layer | When to use |
|--------|-----------|-------------|
| `MONGO_URI` only | In-process memory | **One machine, one process** (default) |
| `MONGO_URI` + `REDIS_URL` | Shared Redis | Multiple servers, same counters |
| `COUNTEX_READ_THROUGH=1` | None (read Mongo) | Minimal infra, low volume |

---

## Three consumption surfaces (choose one)

### 1. Primitives (library-first)

Import pure functions and wire storage yourself:

```typescript
import {
  classifyRecord,
  classifyPipeline,
  buildDedupKey,
  MemoryHotStore,
  MongoDurableStore,
  PipelinesEngine,
} from "@x12i/countex-pipelines";
// or: import from "@x12i/countex-pipelines/primitives"
```

### 2. Embedded facade (in-process)

```typescript
import { createCountexFromEnv } from "@x12i/countex-pipelines";
// or: import from "@x12i/countex-pipelines/client"
```

### 3. HTTP API (optional separate package)

Run as a standalone service for remote or non-Node consumers:

```bash
npm install @x12i/countex-pipelines-server
MONGO_URI=... countex-pipelines-server
```

See [`@x12i/countex-pipelines-server`](../countex-pipelines-server/README.md).

Ingest is **write-through to Mongo** — HTTP `200` means durably recorded.

---

## Counter model

| Field | Role |
|-------|------|
| `tenantId` | Isolation scope |
| `subject` | Host-defined subject type |
| `recordId` | Item being processed |
| `pipelineId` | Host-defined processor / workflow id |
| `dimensions` | Optional tags for rollup axes |
| **Metric** | `succeeded` \| `pending` \| `failed` \| `running` (derived) |

## Classification (per record)

| Bucket | Rule |
|--------|------|
| **succeeded** | At least one pipeline `done`, and **no** pipeline `failed` |
| **failed** | At least one pipeline `failed` (wins over done) |
| **pending** | No terminal status yet |
| **running** | Per-pipeline: latest status is `in_progress` |

---

## Starting from the middle

1. **Redis/memory empty, Mongo has counters** → `initialize({ tenantId, fromCounterStore: true })` reloads and warms the hot layer.
2. **Counters never existed, host has history** → implement `CountexRebuildSource.scan()` and call `initialize({ tenantId, fromSource })` (in-process only).
3. **Process restart** → same as (1); Mongo is source of truth.

---

## Install

```bash
npm install @x12i/countex-pipelines mongodb
# optional for multi-server:
npm install ioredis
```

Node 18+.

## Docs

- [Use-case playbook](docs/use-cases/processing-pipeline-counters.md)

## License

MIT

---
_Source: https://npm.io/package/@x12i/countex-pipelines · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._