# @dxos/pipeline

> Generic back-pressured streaming pipeline (source → stages → sink).

Latest version **0.16.0** (published 2026-10-19) · FSL-1.1-Apache-2.0 license · 0 weekly downloads

## Install

```sh
npm install @dxos/pipeline
pnpm add @dxos/pipeline
yarn add @dxos/pipeline
bun add @dxos/pipeline
```

## Health

**Score 70/100 (B)** — status: active.

Positive: has types; esm support; no vulnerabilities; has provenance; recently updated; high maintenance score.

Warnings: low downloads; pre 1.0.

## Facts

| | |
|---|---|
| Version | 0.16.0 |
| Published | 2026-10-19 |
| First published | 2026-07-06 |
| Weekly downloads | 0 |
| License | FSL-1.1-Apache-2.0 |
| TypeScript types | bundled |
| Module format | ESM |
| Dependencies | 3 |
| Unpacked size | 307.1 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| Provenance | attested (GitHub Actions) |
| GitHub stars | 531 |
| Author | info@dxos.org |
| Maintainers | richburdon, marik_d, wittjosiah, dxos-bot, mykola-vrmchk |

## Links

- npm: https://www.npmjs.com/package/@dxos/pipeline
- Repository: https://github.com/dxos/dxos
- Homepage: https://dxos.org
- Issues: https://github.com/dxos/dxos/issues
- npm.io page: https://npm.io/package/@dxos/pipeline

## Dependencies (3)

- [@dxos/log](https://npm.io/package/@dxos/log.md) 0.16.0
- [@dxos/progress](https://npm.io/package/@dxos/progress.md) 0.16.0
- [@dxos/invariant](https://npm.io/package/@dxos/invariant.md) 0.16.0

## Recent versions

- 0.16.0 (latest) — 2026-10-19
- 0.17.0-next-6a61a88494afb051ad9f12d6eedd1479c54b0cea (next) — 2026-11-18
- 0.16.0-next-60712ab191a99ba90316cc4be753e9e11b17100a — 2026-10-19
- 0.16.0-next-b440559823fb87ed1a4590dab21ae924d4d76c3a — 2026-10-19
- 0.14.0 — 2026-10-10
- 0.13.0 — 2026-10-07
- 0.12.0-next-73daef452f1464b62efdbca75fb0067c1066f264 — 2026-08-27
- 0.12.0-next-2800d0376abcc62d5544afdd744b16b2b960434e — 2026-08-27
- 0.12.0-next-2be9f24c465d98f4bd186aa03ece788902e6c17a — 2026-08-27
- 1.0.0-next-bbabc02363f18d7643ea4160e3a7a7809870fd0d — 2026-08-10
- 0.11.1 — 2026-08-05
- 0.11.0 — 2026-08-05
- 0.9.0 — 2026-07-13
- 0.0.0 — 2026-07-06

## README

# @dxos/pipeline

Generic, back-pressured streaming pipeline. A pipeline runs an Effect `Stream` source through an
ordered chain of stages — each a pipeable `Stream → Stream` transform, composed with `pipe` — and
drains the result to a sink. Back pressure is the default (pull-based `Stream`) and configurable
per pipeline or per stage (`suspend` / `sliding` / `dropping`). Shared dependencies flow through
Effect's Requirements channel (`Context.Tag` / `Layer`), provided once at the edge.

## Usage

```ts
import * as Effect from 'effect/Effect';
import * as Stream from 'effect/Stream';

import { EffectEx } from '@dxos/effect';
import { Pipeline, Stage } from '@dxos/pipeline';

await EffectEx.runPromise(
  Stream.fromIterable([1, 2, 3]).pipe(
    Stage.map('double', (n: number) => Effect.succeed(n * 2)),
    Pipeline.run({ sink: (out) => Effect.sync(() => console.log(out)) }),
  ),
);
```

Shared dependencies flow through Effect's Requirements channel rather than an explicit `context`
parameter — a stage reads them with `yield*` inside its `Effect.gen`, and the caller provides them
once at the edge:

```ts
class Factor extends Context.Tag('Factor')<Factor, { readonly factor: number }>() {}

const program = source.pipe(
  Stage.map('scale', (n: number) => Factor.pipe(Effect.map(({ factor }) => n * factor))),
  Pipeline.run({ sink }),
);

await EffectEx.runPromise(Effect.provide(program, Layer.succeed(Factor, { factor: 5 })));
```

See [`@dxos/pipeline-email`](../pipeline-email) for a worked pipeline over the Enron email dataset.

---
_Source: https://npm.io/package/@dxos/pipeline · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._
