complex-filter-stream v0.1.4
complex-filter-stream
Filter out objects that don't match a series of filtering requirements.
TL;DR
Basic usage
const complexFilterStream = require('complex-filter-stream')
const readableStream = getaReadableObjectModeStreamSomeHow()
const queryStack = [
{
"type": "filter",
"not": false,
"searchFor": "chocolate",
"searchOn": "flavor",
"searchType": "contains"
}
]
const myFilter = complexFilterStream(queryStack)
readableStream.pipe(myFilter)
How to build the queryStack.
complex-filter-stream
is a Transform
stream made out of a bundle of transform streams built based on a series of filtering requirements. They are passed to the stream as a json object.
A simple filter
[
{
"type": "filter",
"not": false,
"searchFor": "search string",
"searchOn": "microscopist",
"searchType": "contains"
}
]
Notice that the filter object
is part of an array. That is because complex-filter-stream
let you stack filters. Now let's go through the specifics.
type
The type
field can accept three values: filter
, AND
and OR
. We will talk extensively about it soon enough.
not
The not
field takes either false
or true
. If not present it defaults to false
.
searchFor
The searchFor
field takes what should be expected to see in the object for filter in.
searchOn
The searchOn
field takes the fields of the object that will be filtered.
searchType
The searchType
field is used to identify the type of filter it should be used. Currently, complex-filter-stream
supports:
contains
: passes if the value of searchFor
matches anywhere in the object[searchOn]
.exact
: passes if the value of searchFor
matches exactly the object[searchOn]
.startsWith
: passes if the value of searchFor
matches the beginning of the object[searchOn]
.endsWith
: passes if the value of searchFor
matches the end of the object[searchOn]
.regex
: passes if the javascript
flavored regex
in searchFor
matches object[searchOn]
.
There is also support for values:
exactValue
: passes if the value of searchFor
is exactly the value of object[searchOn]
.lessThan
: passes if the value of object[searchOn]
is smaller than searchFor
.greaterThan
: passes if the value of object[searchOn]
is larger than searchFor
.between
: passes if the value of object[searchOn]
is in between two semi-colon separated values in searchFor
.
In this cases, NOT also works and invert the search intervals if
true
A complex filter.
Let's suppose that our objects to be filtered looks like this:
{
"date": 1161302400,
"NBCItaxID": 80880,
"speciesName": "Hylemonella gracilis",
"tiltSingleDual": 1,
"defocus": -12,
"dosage": 75,
"tiltConstant": 1,
"tiltMin": -63,
"tiltMax": 60,
"tiltStep": 0.9,
"microscopist": "Gavin Murphy",
"institution": "Caltech",
"lab": "Jensen Lab",
"sid": "gm2006-10-20-4"
}
and we would like to filter in all objects with NCBItaxID
as 80880
AND microscopist
as Gavin Murphy
.
To do that we need two filter requirements, let's see how to organize them:
[
{
"type": "filter",
"searchFor": 80880,
"searchOn": "NCBItaxID",
"searchType": "exact",
},
{
"type": "filter",
"searchFor": "Gavin Murphy",
"searchOn": "microscopist",
"searchType": "exact"
}
]
so... the default association in complex-filter-stream
is AND
. As we add type:filter
objects to the stack, they will be considered in sequence as in AND
form.
Now, let's build a filter for objects with NCBItaxID
as 80880
AND microscopist
as Gavin Murphy
OR Matt Swulius
:
[
{
"type": "filter",
"searchFor": 80880,
"searchOn": "NCBItaxID",
"searchType": "exact",
},
{
"type": "OR",
"args": [
{
"type": "filter",
"searchFor": "Gavin Murphy",
"searchOn": "microscopist",
"searchType": "exact"
},
{ "type": "filter",
"searchFor": "Matt Suwlius",
"searchOn": "microscopist",
"searchType": "exact",
}
]
}
]
As we can see complex-filter-stream
uses a different type
of filter object to listen to associations.
Now, let's build a filter for objects with NCBItaxID
as 80880
AND microscopist
as Gavin Murphy
OR Matt Swulius
, but from all the objects with Matt Swulius
let's only filter in the ones with lab:"Jensen Lab"
:
[
{
"type": "filter",
"searchFor": 80880,
"searchOn": "NCBItaxID",
"searchType": "exact",
},
{
"type": "OR",
"args": [
{
"type": "filter",
"searchFor": "Gavin Murphy",
"searchOn": "microscopist",
"searchType": "exact"
},
{
"type": "AND",
"args": [
{ "type": "filter",
"searchFor": "Matt Suwlius",
"searchOn": "microscopist",
"searchType": "exact",
},
{ "type": "filter",
"searchFor": "Jensen Lab",
"searchOn": "lab",
"searchType": "exact",
},
]
}
]
}
]
In this case, we had to declare another type of association (AND
). So as a rule of thumb, everytime that there is a change in association type (AND
or OR
), a new type
of association must be declared.
If we would like to make the lab
requirement to all of them:
[
{
"type": "filter",
"searchFor": 80880,
"searchOn": "NCBItaxID",
"searchType": "exact",
},
{ "type": "filter",
"searchFor": "Jensen Lab",
"searchOn": "lab",
"searchType": "exact",
},
{
"type": "OR",
"args": [
{
"type": "filter",
"searchFor": "Gavin Murphy",
"searchOn": "microscopist",
"searchType": "exact"
},
{ "type": "filter",
"searchFor": "Matt Suwlius",
"searchOn": "microscopist",
"searchType": "exact",
}
]
}
]