qflatfile v0.0.3

qFlatFile is Flat File parser, writer and converter for node.js services:

var qff = require('qflatfile');
var qffp = qff.getParser({delim: ','})

var sFlatFileString = ""
// Load flat file into a sFlatFileString

var aData = qffp.parse(sFlatFileString)
// aData contains an Array containing the parsed contents from flat file

This library is written to be executed syncronously. As such, it is not recommended to be used in usecases where the data processed is very long. In such cases It would be best to use flat file services that run Asyncronously.

Current Status

This is a first release of a code base that was ported from another language and proved to be very stable in a production environment with millions of cycles. Its not to say that this code will work 100% of the time but its quite solid. If any failure modes are found, please let me know and I will be interested to get fixes done.

Installation

npm install qflatfile

Features

• read any properly formed flat file (csv, tsv, etc.)
• write properly formatted flat files
• convert json file over to a flat file (csv, tsv, etc.)

Using the parser

The parser will convert a string of a properly formed flat file in almost any format (comma, tab etc.)

var qff = require('qflatfile');
var qffp = qff.getParser({delim: ','})

var sFlatFileString = ""
// Load flat file into a sFlatFileString

var aData = qffp.parse(sFlatFileString)
// aData contains an Array containing the parsed contents from flat file

For qff.getParser({<opts>}) the following options are available

Using the writer

The writer will convert a Array to a string of a properly formed flat file in almost any format (comma, tab etc.)

var qff = require('qflatfile');
var qffw = qff.getWriter({delim: ','})

var aTable = []
// Load in a 2d Array of data to be sent to a flat file

var sFlatFile = qffw.writeTable(aTable)
// sFlatFile contains a properly formed Flat file of the data

// Alternatively, you can write one record at a time
var aRecord = []
// Load in a single record into aRecord

var sFFRec = qffw.writeRecord(aRecord)
// sFFRec contains a properly formed Flat file record

For qff.getWriter({<opts>}) the following options are available

Using the converter

The converter will take a file containing json and will output into a flat file.

var qff = require('qflatfile');
var qffc = qff.getConverter({
	 delim: '\t'
	,jsonIn: true
	,asFileIn: true
	,asFileOut: true
	,fileInName: './testFileIn.json'
	,fileOutName: './flatFileOut.tsv'
	,nullValue: '-'
	,ignoreFields: ["accountId","errorDetails","folderId","id","broadcastMessageId","errorDetails","type"]
	,jsonProcessor: function(r){
		d=new Date(r.timeStamp);
		r.creationDate = new Date(r.creationDate);
		r.date = d.getFullYear()+"/"+d.getMonth()+"/"+d.getDay();
		r.dateTime = d.getHours()+":"+d.getDate();
		return r
	}
});

qffc.convertSync();
// After execution, output file should be generated with properly formatted 
// flat file

For qff.getConverter({<opts>}) the following options are available

Future

• Converter: CLI tool Conversion of flat file in and JSON file output conversion of flat file in and JSON String out Conversion of flat file string in and JSON file out * Conversion of flat file string in and JSON string out
• Parser: * Allow processing of an input file as well
• writer: * Allow writing to an output file

License

BSD 2 Clause. See "LICENSE.txt"