Web-stream-util NPM

Web-stream-util

Utilities for working with web streams, including utilities for light-weight stream transforms modeled on the Reader interface.

With ReadableStream now available in ServiceWorkers, streaming transformation & composition of HTML has become an attractive option for server-side and client-side low-latency page composition. To this end, this project provides utilities for working with streams and Reader objects.

A Reader as defined in the stream spec has mainly two methods:

read(): Promise<ResultObject>, with ResultObject an object with a done boolean & value mixed property. If done is false, value is undefined. Once done is true, value is undefined & the stream is exhausted.
cancel(reason): Cancel the stream.

These are the main methods, but there are also releaseLock() and a closed getter.

Reader transforms

This library defines some utilities for working with transforms modeled on this Reader interface. One such transform for HTML is web-html-stream, a streaming HTML element match & transform exposing a Reader interface. Its HTMLTransformReader constructor is passed an input ReadableStream, Reader, or an Array that can be adapted to a stream:

return fetch('http://example.com')
.then(response => {
    // Transform the incoming HTML
    const htmlReader = new HTMLTransformReader(response.body, transformOptions);
    // Convert Reader to a ReadableStream using Web-stream-util's toStream()
    return webStreamUtil.toStream(htmlReader);
});

Since each transform implements a Reader interface & takes a Reader as input, it is fairly easy to build up transform chains. See this demo ServiceWorker for an example of more interesting transform chains.

API

`toReader(input: Reader|ReadableStream|Array|String|Buffer): Reader`

Adapts a variety of input types to a Reader.

`toStream(input: toReader-able): ReadableStream`

Adapts a variety of Reader-convertible types to a ReadableStream. The ReadableStream constructor is expected to be available in the global environment.

`readToArray(input: toReader-able): Promise<Array>`

Drains anything that can be adapted to a Reader to an Array.

`readToString(input: toReader-able): Promise<String>`

Drains anything that can be adapted to a Reader to a string. Converts Buffer or ArrayBuffers to String using a TextDecoder instance.

`new TextDecodeReader(input: toReader-able): Reader<String>`

Decodes a stream of Buffer or ArrayBuffers to a string stream, using a TextDecoder.

`new TextEncodeReader(input: toReader-able): Reader<Buffer|ArrayBuffer>`

Encodes a stream of Strings into a stream of Buffers (node) or ArrayBuffers (browser).

`new FlatStreamReader(input: toReader-able, ctx): Reader`

Flattens / evaluates a stream of complex objects into plain values like Strings and Buffers. Evaluation rules:

function f: Call f(ctx), evaluate result.
Promise: Wait for Promise to resolve, evaluate result.
Reader: Splice in / read sub-reader, and evaluate each returned value.
ReadableStream: Splice in / read sub-stream, and evaluate each returned value.
String, Buffer, Number etc: Pass through.

In a typical streaming composition pipeline, this transform is placed at the end of the pipeline, producing a single, flattened stream of values. An especially interesting use is the pre-compilation of templates into arrays of strings and functions, which can then be efficiently evaluated with FlatStreamReader at runtime.

web-stream-util v0.3.7