0.2.0 • Published 7 years ago
kcl-readable-stream v0.2.0
kcl-readable-stream
This project convert a KCL object into a readable stream.
There are three (optional) options that can be used in the constructor:
kcl.initialize
: Will be called withdata
andcallback
when the KCL subprocess is initialized. The callback function must be called.kcl.shutdown
: Will be called withdata
andcallback
when the KCL subprocess is shut down because of an error or if the process is stopped. The callback function must be called.kcl.streamSingleRecords
: Iftrue
,data
events will be emitted with single records; otherwise, an array of records will be used. Defaults tofalse
.
Example usage
// run this with something like:
// `kcl-bootstrap --java /usr/bin/java -e -p ./conf/kinesis.properties`
const kcl = require('aws-kcl');
const KclReadableStream = require('kcl-readable-stream');
const kclProcessor = new KclReadableStream({
kcl: {
initialize: (data, callback) => {
console.log('Initialized!');
callback();
},
shutdown: (data, callback) => {
console.log('Shutdown!');
callback();
},
streamSingleRecords: false
}
});
// if !streamSingleRecords
kclProcessor.on('data', (records) => {
console.log('This is an array of records: ', records);
});
// if streamSingleRecords === true
kclProcessor.on('data', (record) => {
console.log('This is a single record: ', record);
});
// you can also .pipe()
kclProcessor.pipe(process.stdout);
kcl(kclProcessor).run();