0.0.10 • Published 2 years ago
streamifi v0.0.10
streamifi
streamifi your backends
It is what it sounds like it is. This package allows you to stream data from various backends.
Usage
// example to stream a file from s3 to stdout line by line
import { s3 } from "../index";
// we highly recommend the use of pipeline that propagates errors unlike pipe. you could still use pipe if you desire so
import { pipeline } from "stream/promises";
import { s3, util_transform } from "../index";
pipeline(
s3.reader(
{
credentials: {
accessKeyId: ...,
secretAccessKey: ...,
},
Bucket: ...,
Key: ...,
},
// codec is used to tell the reader how to break the stream into chunks/msgs
// in the below line, we are asking the reader to break the stream into lines with each line being delimited by a new line char (\n)
[
{
codec: "line",
codecOptions: { lineDelimiter: "\n" }
}
// multiple codecs can be specified allowing you to perform things like JSONParse per line
]
),
// util_transform is a utility function provided that help create functions that transform a stream
util_transform([
(x) => {
console.log(x);
},
])
).catch((e) => { // this is how error handling is performed
console.error("ERROR", e);
});
Backends
Various backends are available (including bring your own backend*).