Qlobber NPM | npm.io

qlobber

Node.js globbing for amqp-like topics.

Note: Version 5.0.0 adds async and worker thread support when used on Node 12+.

Example:

var assert = require('assert');
var Qlobber = require('qlobber').Qlobber;
var matcher = new Qlobber();
matcher.add('foo.*', 'it matched!');
assert.deepEqual(matcher.match('foo.bar'), ['it matched!']);
assert(matcher.test('foo.bar', 'it matched!'));

The API is described here.

qlobber is implemented using a trie, as described in the RabbitMQ blog posts here and here.

Installation

npm install qlobber

Another Example

A more advanced example using topics from the RabbitMQ topic tutorial:

var assert = require('assert');
var Qlobber = require('qlobber').Qlobber;
var matcher = new Qlobber();
matcher.add('*.orange.*', 'Q1');
matcher.add('*.*.rabbit', 'Q2');
matcher.add('lazy.#', 'Q2');
assert.deepEqual(['quick.orange.rabbit',
                  'lazy.orange.elephant',
                  'quick.orange.fox',
                  'lazy.brown.fox',
                  'lazy.pink.rabbit',
                  'quick.brown.fox',
                  'orange',
                  'quick.orange.male.rabbit',
                  'lazy.orange.male.rabbit'].map(function (topic)
                  {
                      return matcher.match(topic).sort();
                  }),
                 [['Q1', 'Q2'],
                  ['Q1', 'Q2'],
                  ['Q1'],
                  ['Q2'],
                  ['Q2', 'Q2'],
                  [],
                  [],
                  [],
                  ['Q2']]);

Async Example

Same as the first example but using await:

const assert = require('assert');
const { Qlobber } = require('qlobber').set_native(require('qlobber-native'));
const matcher = new Qlobber.nativeString();

(async () => {
    await matcher.addP('foo.*', 'it matched!');
    assert.deepEqual(await matcher.matchP('foo.bar'), ['it matched!']);
    assert(await matcher.testP('foo.bar', 'it matched!'));
})();

Worker Thread Example

Same again but the matching is done on a separate thread:

const { Qlobber } = require('qlobber').set_native(require('qlobber-native'));
const {
    Worker, isMainThread, parentPort, workerData
} = require('worker_threads');

if (isMainThread) {
    const matcher = new Qlobber.nativeString();
    matcher.add('foo.*', 'it matched!');
    const worker = new Worker(__filename, {
        workerData: matcher.state_address
    });
    worker.on('message', msg => {
        const assert = require('assert');
        assert.deepEqual(msg, [['it matched!'], true]);
    });
} else {
    const matcher = new Qlobber.nativeString(workerData);
    parentPort.postMessage([
        matcher.match('foo.bar'),
        matcher.test('foo.bar', 'it matched!')
    ]);
}

Licence

MIT

Tests

qlobber passes the RabbitMQ topic tests (I converted them from Erlang to Javascript).

To run the tests:

npm test

Lint

grunt lint

Code Coverage

npm run coverage

c8 results are available here.

Coveralls page is here.

Benchmarks

grunt bench

qlobber is also benchmarked in ascoltatori.

Native Qlobbers

The Javascript Qlobbers don't support asynchronous calls and worker threads because Javascript values can't be shared between threads.

In order to support asynchronous calls and worker threads, a native C++ implementation is available in the qlobber-native module.

Add qlobber-native as a dependency to your project and then add it to qlobber like this:

require('qlobber').set_native(require('qlobber-native'));

Note that set_native returns qlobber's exports so you can do something like this:

const { Qlobber } = require('qlobber').set_native(require('qlobber-native'));

Note that qlobber-native requires Gnu C++ version 9+ and Boost 1.70+, including the boost_context runtime library.

Once's you've added it to qlobber, the following classes will be available alongside the Javascript classes:

Qlobber.nativeString
Qlobber.nativeNumber
QlobberDedup.nativeString
QlobberDedup.nativeNumber
QlobberTrue.native

They can only hold values of a single type (currently strings or numbers).

Asynchronous calls

The native classes support the same API as the Javascript classes but have the following additional methods:

addP
removeP
matchP
match_iterP
testP
clearP
visitP
get_restorerP

They correspond to their namesakes but return Promises. Note that match_iterP and visitP return async iterators.

API

Source: lib/qlobber.js

Qlobber(options)

Creates a new qlobber.

Parameters:

{Object} [options] Configures the qlobber. Use the following properties:
- {String} separator The character to use for separating words in topics. Defaults to '.'. MQTT uses '/' as the separator, for example.
- {String} wildcard_one The character to use for matching exactly one non-empty word in a topic. Defaults to '*'. MQTT uses '+', for example.
- {String} wildcard_some The character to use for matching zero or more words in a topic. Defaults to '#'. MQTT uses '#' too.
- {Boolean} match_empty_levels If true then wilcard_one also matches an empty word in a topic. Defaults to false.
- {Boolean|Map} cache_adds Whether to cache topics when adding topic matchers. This will make adding multiple matchers for the same topic faster at the cost of extra memory usage. Defaults to false. If you supply a Map then it will be used to cache the topics (use this to enumerate all the topics in the qlobber).
- {Integer} cache_splits How many topic.split results to cache. When you pass in a topic, it has to be split on the separator. Caching the results will make using the same topics multiple times faster at the cost of extra memory usage. Defaults to 0 (no caching). The number of split results cached is limited by the value you pass here.
- {Integer} max_words Maximum number of words to allow in a topic. Defaults to 100.
- {Integer} max_wildcard_somes Maximum number of wildcard_some words in a topic. Defaults to 3.

Go: TOC

Qlobber.prototype.add(topic, val)

Add a topic matcher to the qlobber.

Note you can match more than one value against a topic by calling add multiple times with the same topic and different values.

Parameters:

{String} topic The topic to match against.
{Any} val The value to return if the topic is matched.

Return:

{Qlobber} The qlobber (for chaining).

Go: TOC | Qlobber.prototype

Qlobber.prototype.remove(topic, val)

Remove a topic matcher from the qlobber.

Parameters:

{String} topic The topic that's being matched against.
{Any} [val] The value that's being matched. If you don't specify val then all matchers for topic are removed.

Return:

{Qlobber} The qlobber (for chaining).

Go: TOC | Qlobber.prototype

Qlobber.prototype.match(topic)

Match a topic.

Parameters:

{String} topic The topic to match against.

Return:

{Array} List of values that matched the topic. This may contain duplicates. Use a QlobberDedup if you don't want duplicates.

Go: TOC | Qlobber.prototype

Qlobber.prototype.match_iter()

Match a topic, returning the matches one at a time.

Return:

{Iterator} An iterator on the values that match the topic. There may be duplicate values, even if you use a QlobberDedup.

Go: TOC | Qlobber.prototype

Qlobber.prototype.test(topic, val)

Test whether a topic match contains a value. Faster than calling match and searching the result for the value. Values are tested using test_values.

Parameters:

{String} topic The topic to match against.
{Any} val The value being tested for.

Return:

{Boolean} Whether matching against topic contains val.

Go: TOC | Qlobber.prototype

Qlobber.prototype.test_values(vals, val)

Test whether values found in a match contain a value passed to test. You can override this to provide a custom implementation. Defaults to using indexOf.

Parameters:

{Array} vals The values found while matching.
{Any} val The value being tested for.

Return:

{Boolean} Whether vals contains val.

Go: TOC | Qlobber.prototype

Qlobber.prototype.clear()

Reset the qlobber.

Removes all topic matchers from the qlobber.

Return:

{Qlobber} The qlobber (for chaining).

Go: TOC | Qlobber.prototype

Qlobber.prototype.visit()

Visit each node in the qlobber's trie in turn.

Return:

{Iterator} An iterator on the trie. The iterator returns objects which, if fed (in the same order) to the function returned by get_restorer on a different qlobber, will build that qlobber's trie to the same state. The objects can be serialized using JSON.stringify, if the values you store in the qlobber are also serializable.

Go: TOC | Qlobber.prototype

Qlobber.prototype.get_restorer(options)

Get a function which can restore the qlobber's trie to a state you retrieved by calling visit on this or another qlobber.

Parameters:

{Object} [options] Options for restoring the trie.
- {Boolean} cache_adds Whether to cache topics when rebuilding the trie. This only applies if you also passed cache_adds as true in the constructor.

Return:

{Function} Function to call in order to rebuild the qlobber's trie. You should call this repeatedly with the objects you received from a call to visit. If you serialized the objects, remember to deserialize them first (e.g. with JSON.parse)!

Go: TOC | Qlobber.prototype

QlobberDedup(options)

Creates a new de-duplicating qlobber.

Inherits from Qlobber.

Parameters:

{Object} [options] Same options as Qlobber.

Go: TOC

QlobberDedup.prototype.test_values(vals, val)

Test whether values found in a match contain a value passed to test. You can override this to provide a custom implementation. Defaults to using has.

Parameters:

{Set} vals The values found while matching (ES6 Set).
{Any} val The value being tested for.

Return:

{Boolean} Whether vals contains val.

Go: TOC | QlobberDedup.prototype

QlobberDedup.prototype.match(topic)

Match a topic.

Parameters:

{String} topic The topic to match against.

Return:

{Set} ES6 Set of values that matched the topic.

Go: TOC | QlobberDedup.prototype

QlobberTrue(options)

Creates a new qlobber which only stores the value true.

Whatever value you add to this qlobber (even undefined), a single, de-duplicated true will be stored. Use this qlobber if you only need to test whether topics match, not about the values they match to.

Inherits from Qlobber.

Parameters:

{Object} [options] Same options as Qlobber.

Go: TOC

QlobberTrue.prototype.test_values()

This override of test_values always returns true. When you call test on a QlobberTrue instance, the value you pass is ignored since it only cares whether a topic is matched.

Return:

{Boolean} Always true.

Go: TOC | QlobberTrue.prototype

QlobberTrue.prototype.match(topic)

Match a topic.

Since QlobberTrue only cares whether a topic is matched and not about values it matches to, this override of match just calls test (with value undefined).

Parameters: