1.0.3 • Published 3 years ago
@rpidanny/streamline.js v1.0.3
streamline.js
A JavaScript package that helps to reads and processes a stream line-by-line in order or in batches in parallel.
Install
$ npm install --save @rpidanny/streamline.jsUsage
Functions
processLine(readStream, handler, concurrency)
| Parameters | Default | Description |
|---|---|---|
readStream | None | Any Node.js ReadStream object |
handler | None | A processing function that is called on every line of the stream with the line as the argument |
concurrency | 1 | The number of concurrent execution of the handler |
processJson(readStream, handler, concurrency)
| Parameters | Default | Description |
|---|---|---|
readStream | None | Any Node.js ReadStream object |
handler | None | A processing function that is called on every line of the stream with the parsed JSON object as the argument |
concurrency | 1 | The number of concurrent execution of the handler |
processCsv(readStream, handler, concurrency)
| Parameters | Default | Description |
|---|---|---|
readStream | None | Any Node.js ReadStream object |
handler | None | A processing function that is called on every line of the stream with the parsed CSV as the argument |
concurrency | 1 | The number of concurrent execution of the handler |
Example
const readStream = fs.createReadStream(`jsonlFile.json`)
await processJson(
readStream,
async (item: Record<string, unknown>) => {
console.log(item)
},
2,
)Classes
Base classes
StreamlineStreamlineCsvStreamlineJson
These are abstract classes that can be inherited to create complex classes to process read streams.
Forked classes
These classes are derived from Base classes which is suitable for some common use cases.
SimpleCsvProcessorSimpleJsonProcessorSimpleCsvProcessor