@e2fyi/streams v1.0.1

@e2fyi/streams

This NodeJS library provides custom NodeJS streams for specific use cases.

Currently, the following streams are available:

• DocumentTagger: A Transform stream to tag an auto-increment field to each object in the stream. Can also mutate the stream objects through a function or default Object.

• MongooseStream: A Transform stream which will bulk write the objects in the stream to mongodb via a mongoose model.

ChangeLog

• v1.0.0: new streams: DocumentTagger, MongooseStream.

Quick start

Using CommonJS module

// importing DocumentTagger and MongooseStream
const {DocumentTagger, MongooseStream} = require('@e2fyi/streams');

API reference

The API documentation is also available at https://e2fyi.github.io/streams.

@e2fyi/streams

• @e2fyi/streams
  • .DocumentTagger ⇐ stream.Transform
    • new DocumentTagger(opts)
  • .MongooseStream ⇐ stream.Transform
    • new MongooseStream(opts)
  • .DocumentTaggerSettings : Object
  • .MongooseStreamSettings : Object

@e2fyi/streams.DocumentTagger ⇐ stream.Transform

Transform Object stream (objectMode=true) to tag with an autoIncrement id. An object or function can be optionally provided to mutate each object in the stream.

Kind: static class of @e2fyi/streams
Extends: stream.Transform
Emits: filtered

new DocumentTagger(opts)

Create a new DocumentTagger stream.

Example

const docTagger = new DocumentTagger({autoIncrement: 'id', mutate: { project: 'test' }});
someReadableStreamFromArray([{text: 'abc'}, {text: 'efg'}])
  .pipe(docTagger)
  .pipe(process.stdout);
// stdout >
// {"text": "abc", "id": 0, "project": "test"}
// {"text": "efg", "id": 1, "project": "test"}

@e2fyi/streams.MongooseStream ⇐ stream.Transform

A custom NodeJS Transform stream to mongo via mongoose.

Kind: static class of @e2fyi/streams
Extends: stream.Transform
Emits: mongoose-bulk-write

new MongooseStream(opts)

Create a Transform stream which bulkWrite to mongo based on the itemWaterMark. model (mongoose Model) is a required field.

Example

var stream2mongo = new MongooseStream({mode: SomeMongooseModel});
someReadableStreamFromArray([{text: 'abc'}, {text: 'efg'}])
  .pipe(stream2mongo) // writes to mongo (while stream are also passthrough)
  .pipe(response); // stream same results back to some request

@e2fyi/streams.DocumentTaggerSettings : Object

Settings for DocumentTagger.

Kind: static typedef of @e2fyi/streams
Properties

@e2fyi/streams.MongooseStreamSettings : Object

Settings for MongooseStream.

Kind: static typedef of @e2fyi/streams
Properties

Development

Unit testing

npm test

Build documentation

npm run build