1.1.3 • Published 2 years ago
@giorgi-g/csv-parser v1.1.3
CSV PARSER ON STEROIDS
CSV files parser on TypeScript with Sequelizer. For easier migration.
Install npm modules
npm install
Then configure a database according to the .env.example file
In config.ts you will also see sample default values...
We can define different options upon reading a file e.g:
interface CSVParserOptions {
mapKeyIndexes?: number[];
mapKeySeparator?: string;
classPath?: string;
classObjectGetterName?: string;
delimiter?: string;
rootDir?: string;
rootEnv?: string;
fileExtension?: string;
mergeMapKeyValues?: boolean;
}
mapKeyIndexes
- After reading a file a
Map<any|number, any>
is generated by default it has numeric index as a key, but we can assign a key generated by the properties. - If we want records to have a custom key from CSV file e.g: email we can set
mapKeyIndexes: [3]
where 3 is the index in CSV for email. - We can also have combined key e.g:
[0, 3]
which will give us${ID}-${EMAIL}
- Note: the keys are separated by the Dash symbol if you want to override the default you can change the value of
mapKeySeparator
key.
- After reading a file a
classPath && classObjectGetterName
- After parsing the data from CSV you can cast it to a certain Class which by default can be created inside entities' folder. e.g:
{ classPath: '../entities/Profile' }
- If you want value of the map to be some certain property or getter inside the Class you can provide a value for classObjectGetterName e.g:
{ classObjectGetterName: 'profile' }
which is the profile property inside the class
- After parsing the data from CSV you can cast it to a certain Class which by default can be created inside entities' folder. e.g:
- delimiter
- As a delimiter your CSV file can contain
comma: ,
symbol if that is not the default for your CSV you can provide a different value defined inside.
- As a delimiter your CSV file can contain
- mergeMapKeyValues
- While setting a map to the key if you want to receive the same key values in an array make this value true
Example:
const fileName = "sample_profiles"; // file name inside /files/FILE_NAME.csv
const options: CSVParserOptions = {
classPath: '../entities/Profile', // Class name if you want to cast result into a class
classObjectGetterName: 'profile', // Name of the property inside the Class which will be the value inside the Map
mapKeyIndexes: [3] // Email as a key of the Map
};
// Initialize the parser with properties
const csvParser = new CSVParser(
fileName,
options,
);
// Read the data inside the parser
csvParser.Read().then((data) => {
data.forEach((profile) => {
console.log('>>> profile', profile);
})
});
const connection = dbConnection("profile", 100, 0).then((response) => {
console.log('>>> response from db:', response);
}).catch((error) => {
console.log('>>> error: ', error);
});
dbConnection = async (schema?: string, limit: number = 0, offset: number = 0) => {
const DB = Sequelizer(schema);
return DB.query(`SELECT uuid, brand_id FROM
profile.profiles
WHERE brand_id IS NOT NULL
ORDER BY uuid DESC LIMIT ${limit} OFFSET ${offset}`,
{
type: QueryTypes.SELECT,
logging: false,
});
}