Timeseriesfilter NPM

TimeSeriesFilter-Node

Parsing of user-customised language and translation into database queries for time series data.

Installation

This is a Node.js module available through the npm registry.

Before installing, download and install Node.js.

To install, use npm install.

npm install timeseriesfilter

Usage

Once installed, import the IFilterField, IOperators and IField interfaces and the Language and Filter classes. Although not necessary, it is recommended to use TypeScript as it adds type-checking and ensures all fields and parameters are fulfilled.

An IOperators variable and at least one IFilterField must be specified to use the module.

The IOperators field contain the operators used by the parser to identify blocks, ranges, lists, logic and equality operations:

is equals to (one-character maximum)
not does not equal to (one-character maximum)
regIs equals to regex (one-character maximum)
regIs does not equal to regex (one-character maximum)
and logical and
or logical or
list list items separator
rangeInc inclusive range [a; b]
rangeLInc left-inclusive range [a; b)
rangeRInc right-inclusive range (a; b]
rangeNonInc non-inclusive range (a; b)
blockOpen open logical block
blockClose close logical block

The IFilterFields contain the specification of the fields parsed by the module:

key is a one-character unique identifier (string)
base is the regular expression describing the possible states of the field without wildcards (character *).
wildcards is the regular expression describing the possible states of the field with wildcards (wildcards are optional, but the field is mandatory).
groups is the regular expression with the capture groups for the field without wildcards. These are used to capture the values and must not be nested.
groupsWildacrds is the regular expression with the capture groups for the field with wildcards (wildcards are optional, but the field is mandatory).

NOTE: Except for the key field, all the others require RegExp strings.

The IField interface contains the information needed to translate a parsed filter into q database query.

field the field name of the filter in the database
join a list of strings or numbers to be interspersed between the captured groups of the filter
zero zero value that substitutes wildcards in ranges, when using is or not operators, or if type is "num"
type one of "num" and "srt" with the former forcing a conversion of the field to a number

The next step is to create the Language object. The constructor requires a list of IFilterFields and one IOperators.

The Language is then used in the constructor of the Filter object.

With the Filter created, strings can now be parsed using the Filter.fromString(parseString: string): FiltersList | null method. The return is a null if the parsing encountered errors, otherwise it returns a list of objects containing the parsed elements. These elements can be blocks, logic operators and filter items. The parsed elements can always be accessed via the public Filter.parsedFilters variable.

Each custom object used in the filters list has a .name method that contains its name and supports the .toString() method to get a proper string representation.

The Filter object itself supports .toString() which returns a stringified version of the parsedFilters variable.

To translate the parsed filters into queries, a map of IFields interfaces must be created. Each IField must be mapped to the key of the field it refers to.

The fields map is then passed to the Filter.toQuery(databaseType: "sql", fields: Map<string, IField>): string method together with the destination database language.

The modules only supports the following database languages for translation:

"sql" for SQL-like languages

Demo

The module contains a short CLI demo to that allows to test the basic functionalities of the models using the languages specifications used in the example below. To run, use demo script or run directly with node timeseriesfilter/dist/demo.js.

Examples

The following examples use a weekday and time field to showcase multiple capturing groups in action regex support.

import {IFilterField, IOperators, IField, Language, Filter} from "timeseriesfilter"

const fieldTime: IFilterField = {
    key: "T",
    base: /\d{1,2}:\d{1,2}:\d{1,2}\.\d{1,3}|\d{1,2}:\d{1,2}:\d{1,2}|\d{1,2}:\d{1,2}|\d{1,2}/,
    wildcards: /[\d*]{1,2}:[\d*]{1,2}:[\d*]{1,2}\.[\d*]{1,3}|[\d*]{1,2}:[\d*]{1,2}:[\d*]{1,2}|[\d*]{1,2}:[\d*]{1,2}|[\d*]{1,2}/,
    groups: /(\d)?(\d):?(\d)?(\d)?:?(\d)?(\d)?\.?(\d)?(\d)?(\d)?/,
    groupsWildcards: /([\d*])?([\d*]):?([\d*])?([\d*])?:?([\d*])?([\d*])?\.?([\d*])?([\d*])?([\d*])?/
}

const fieldWeekday: IFilterField = {
    key: "W",
    base: /[1-7]/,
    wildcards: /[1-7]/,
    groups: /([1-7])/,
    groupsWildcards: /([1-7])/,
}

const operators: IOperators = {
    is: /=/,
    not: /!/,
    regIs: /~/,
    regNot: /\^/,
    and: /&/,
    or: /\|/,
    list: /,/,
    rangeInc: /-/,
    rangeLInc: />/,
    rangeRInc: /</,
    rangeNonInc: /<>/,
    blockOpen: /\(/,
    blockClose: /\)/,
}

const fields: Map<string, IField> = new Map<string, IField>([
    ["T", {field: "TIME", join: ["", ":", "", ":", "", ".", ""], zero: "0", type: "str"} as IField],
    ["W", {field: "DAY", join: [""], zero: "0", type: "num"} as IField],
])

const language = new Language([fieldTime, fieldWeekday], operators)
const filter = new Filter(language)

filter.fromString("(T=10>18&W=1-4)|(T=10>14&W=5)")
// BlockOpen
// {T Is RangeLInc [["1","0","*","*","*","*","*","*","*"],["1","8","*","*","*","*","*","*","*"]]} And 
// {W Is RangeInc [["1"],["4"]]}
// BlockClose Or BlockOpen
// {T Is RangeLInc [["1","0","*","*","*","*","*","*","*"],["1","4","*","*","*","*","*","*","*"]]} And 
// {W Is ListOr [["5"]]}
// BlockClose
filter.toQuery("sql", fields)
// ( ( TIME >= "10:00:00.000" AND TIME < "18:00:00.000" ) AND ( DAY >= 1 AND DAY <= 4 ) )
// OR
// ( ( TIME >= "10:00:00.000" AND TIME < "14:00:00.000" ) AND ( DAY LIKE 5 ) )
filter.toQuery("mongo", fields)
// {"$or":[{"TIME":{"$lt":"18:00:00.000"},"DAY":{"$lte":4}},{"TIME":{"$lt":"14:00:00.000"},"DAY":5}]}

filter.fromString("T=10-10:00:01.5")
// {T Is RangeInc [["1","0","*","*","*","*","*","*","*"],["1","0","0","0","0","1","5","*","*"]]}
filter.toQuery("sql", fields)
// TIME >= "10:00:00.000" AND TIME <= "10:00:01.500" )

filter.fromString("T~10,11")
// {T RegIs ListOr [["1","0","*","*","*","*","*","*","*"],["1","1","*","*","*","*","*","*","*"]]}
filter.toQuery("sql", fields)
// ( TIME LIKE "10:%%:%%.%%%" OR TIME LIKE "11:%%:%%.%%%" )
filter.toQuery("mongo", fields)
// {"$or":[{"TIME":{"$regex":"10:**:**.***"}},{"TIME":{"$regex":"11:**:**.***"}}]}

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago