# flumeview-query

> map-filter-reduce queries for flumedb

Latest version **8.0.0** (published 2020-06-15) · MIT license · 0 weekly downloads

## Install

```sh
npm install flumeview-query
pnpm add flumeview-query
yarn add flumeview-query
bun add flumeview-query
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 8.0.0 |
| Published | 2020-06-15 |
| First published | 2016-11-22 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 7 |
| Unpacked size | 37.7 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 17 |
| Author | Dominic Tarr |
| Maintainers | arj03, christianbundy, dominictarr, regular |

## Links

- npm: https://www.npmjs.com/package/flumeview-query
- Repository: https://github.com/flumedb/flumeview-query
- Issues: https://github.com/flumedb/flumeview-query/issues
- npm.io page: https://npm.io/package/flumeview-query

## Dependencies (7)

- [deep-equal](https://npm.io/package/deep-equal.md) ^1.0.1
- [pull-stream](https://npm.io/package/pull-stream.md) ^3.4.0
- [pull-flatmap](https://npm.io/package/pull-flatmap.md) 0.0.1
- [pull-paramap](https://npm.io/package/pull-paramap.md) ^1.1.3
- [flumeview-level](https://npm.io/package/flumeview-level.md) ^4.0.3
- [map-filter-reduce](https://npm.io/package/map-filter-reduce.md) ^3.2.0
- [pull-sink-through](https://npm.io/package/pull-sink-through.md) 0.0.0

## Recent versions

- 8.0.0 (latest) — 2020-06-15
- 7.2.2 — 2020-05-25
- 7.2.1 — 2019-07-05
- 7.2.0 — 2019-07-01
- 7.1.1 — 2018-11-26
- 7.1.0 — 2018-10-14
- 7.0.0 — 2018-09-26
- 6.3.0 — 2018-08-13
- 6.2.0 — 2018-05-17
- 6.1.4 — 2018-05-17
- 6.1.2 — 2018-05-14
- 6.1.1 — 2018-05-09
- 6.1.0 — 2018-04-30
- 6.0.1 — 2018-03-25
- 6.0.0 — 2018-03-19
- … 10 more at https://npm.io/package/flumeview-query/versions

## README

# flumeview-query

A [flumeview](https://github.com/flumedb/flumedb) with [map-filter-reduce](https://github.com/dominictarr/map-filter-reduce) queries

## Motivation

This particular module was because I needed to query things
in [secure-scuttlebutt](https://github.com/ssbc/secure-scuttlebutt)
in a flexible way. A previous exploration of this general
idea was [mynosql](https://www.npmjs.com/package/mynosql)
Yes the joke is that it's sql, but for no sql. SQL is actually
totally functional (just with weird names)

map, filter, reduce == select, where, group-by !

Except with none of those icky schemas that just get you down!
(anyway, with ssb we really _can't_ enforce schemas (because
of privacy oriented decentralization))

## Example

``` js
var db = Flume(log).use('query', FlumeQuery(null, {indexes:indexes}))

//write a batch of data to the log
db.append([{
  foo: true,
  bar: 5,
  nested: {baz: 'okay'}
},
{
  foo: false,
  bar: 6,
  nested: {baz: 'okay'}
},
{
  foo: false,
  bar: 7,
  nested: {baz: 'not-okay'}
},
], function (err) {
  //filter for all records which match the above query
  pull(
    db.query.read({query: [
      {$filter: {nested: {baz: 'okay'}}}
    ]}),
    pull.collect(function (err, ary) {
      console.log(ary)
      //out puts the first and second items inserted above.
    })
  )
})
```

## new api

flumeview-query shouldn't hold the indexes, it should just know what
to do with them. Instead of passing in the indexes to create,
pass in an index which can be used. that index would need to 
expose what paths it indexes, of course.

```
query.add(index, createStream) //add an index
```

## Indexes

The indexes argument is an array of indexes that flumeview-query will
be able to look at to do a fast query.

Indexes is an object with the a short name of the index (this will be
stored with every record, so say 3 chars, is recommended) and a value
with the fields being indexed.

For example, `ssb-query`'s indexes look like:

``` js
[
  {key: 'log', value: ['timestamp']},
  {key: 'clk', value: [['value', 'author'], ['value', 'sequence']] },
  {key: 'typ', value: [['value', 'content', 'type'], ['timestamp']] },
  {key: 'cha', value: [['value', 'content', 'channel'], ['timestamp']] },
  {key: 'aty', value: [['value', 'author'], ['value', 'content', 'type'], ['timestamp']]}
]
```

Indexes can be of a single field or multiple fields (which are called
"compound indexes"). Each item in an index must be unique, that is why
the most of the indexes end in `timestamp`, author:sequence is also
unique in ssb, so that index doesn't need timestamp.  The uniqueness
is not enforced by flumeview-query, it is the responsibilty of the
index designer.

Compound indexes optimize queries with multiple fields. For example a
query like: "all posts by @bob" which is `{value: {author: @bob,
content: {type: 'post'}}}` uses the `clk` index.

If a query matches all fields in the index, the query will return 1
item (or zero if there is no record with those values), otherwise
results will be returned in the order of the index.

Properties in the index are used from left to right, a query for
"messages from @bob received since yesterday" `{author: @bob,
timestamp: {$gt: yesterday}}` cannot use author and timestamp fields
in `aty` as it leaves a gap in the `value.content.type` field. This
query with these indexes would use part of a compound index, (clk,
because it matches first) read all the messages by `@bob` and filter
out the records matching the other query parameter (`timestamp: {$gt:
yesterday}`). This is called a "partial scan". A partial scan is
clearly less efficient than matched index, but not as bad as a full
scan (which reads the entire database!)

Queries with compound indexes will end up sorted by the last index
matched. Therefor, put the fields you expect to be exact first!

## Queries

This module uses
[map-filter-reduce](https://github.com/dominictarr/map-filter-reduce)
queries, if the filter stage uses fields that are in a index, then
`select` can choose the best index and perform many queries very
quickly.

See [map-filter-reduce](https://github.com/dominictarr/map-filter-reduce) for documentation of the syntax,
for example queries, performed on top of [secure-scuttlebutt](https://github.com/ssbc/secure-scuttlebutt)

## api : flumedb.use("query", FlumeViewQuery(version, opts))

`version` must be an number. When you update any options,
change the version and the index will rebuild.
`opts` is the options. in particular `{indexes: [...]}` is mandatory.

Here we use the name "query", you can use any name.

## api : FlumeViewQuery(version, {indexes, filter, map}) => FlumeView: query

as required by every flumeview, `version` is a integer. change this
when the indexes or other settings change and the view will be rebuilt.
`indexes` is mandatory. Indexes are the paths supported.
`indexes` to use. `links` is an optional function used for mapping a
value into one or more values for the index. `version` must be an
number. When you change `indexes` or `links`, bump the version and the
index will rebuild.

### query.read({query:MFR_query, limit, reverse, live, old})

Perform the query! limit, reverse, live, old are stardard as with
[other flume streams](https://github.com/flumedb/flumelog-offset).
`unlinkedValues` is an option that can be used to
include the values not part of the index in the return value.

### query.explain ({query:MFR_query, limit, reverse, live, old}) => obj

Figure out what indexes are best to use to perform a query, but do not
actually run the query! If a query is slow or doesn't seem to be
working right, this method can be used to understand what is really
going on. If the return value is `{scan: true}` that means no indexes
are being used. If an index is selected, that should mean it's more
efficient, but it might still be filtering the output.



## License

MIT

---
_Source: https://npm.io/package/flumeview-query · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._