Bulksearch NPM

BulkSearch

Superfast, lightweight and read-write optimized full text search library.

When it comes to the overall speed, BulkSearch outperforms every searching library out there and also provides flexible search capabilities like multi-word matching, phonetic transformations or partial matching. It is basically based on how a hdd manage files on the filesystem. Adding, updating or removing items are also fast as searching for them, but also requires some additional amount of memory. When your index don't needs to be updated continuously then FlexSearch may be a better choice. BulkSearch also provides you a asynchronous processing model to perform queries in background.

Benchmark:

Comparison: https://jsperf.com/compare-search-libraries
Detailed: https://jsperf.com/bulksearch

Supported Platforms:

Browser
Node.js

Supported Module Definitions:

AMD (RequireJS)
CommonJS (Node.js)
Closure (Xone)
Global (Browser)

All Features:

Installation

HTML / Javascript

<html>
<head>
    <script src="js/bulksearch.min.js"></script>
</head>
...

Note: Use bulksearch.min.js for production and bulksearch.js for development.

Use latest from CDN:

<script src="https://cdn.rawgit.com/nextapps-de/bulksearch/master/bulksearch.min.js"></script>

Node.js

npm install bulksearch

In your code include as follows:

var BulkSearch = require("bulksearch");

Or pass in options when requiring:

var index = require("bulksearch").create({/* options */});

AMD

var BulkSearch = require("./bulksearch.js");

Compare BulkSearch vs. FlexSearch

API Overview

Global methods:

BulkSearch.create(\<options>)
BulkSearch.addMatcher({KEY: VALUE})
BulkSearch.register(name, encoder)
BulkSearch.encode(name, string)

Index methods:

Index.add(id, string)
Index.search(string, \<limit>, \<callback>)
Index.search(string, \<page>, \<callback>)
Index.search(options, \<callback>)
Index.update(id, string)
Index.remove(id)
Index.reset()
Index.destroy()
Index.init(\<options>)
Index.optimize()
Index.info()
Index.addMatcher({KEY: VALUE})
Index.encode(string)

Usage

Create Index

BulkSearch.create(\<options>)

var index = new BulkSearch();

alternatively you can also use:

var index = BulkSearch.create();

Create index with custom options

var index = new BulkSearch({

    // default values:

    type: "integer",
    encode: "icase",
    boolean: "and",
    size: 4000,
    multi: false,
    strict: false,
    ordered: false,
    paging: false,
    async: false,
    cache: false
});

Read more: Phonetic Search, Phonetic Comparison, Improve Memory Usage

Add items to an index

Index.add(id, string)

index.add(10025, "John Doe");

Search items

Index.search(string|options, \<limit|page>, \<callback>)

index.search("John");

Limit the result:

index.search("John", 10);

Perform queries asynchronously:

index.search("John", function(result){
    
    // array of results
});

Pass parameter as an object:

index.search({

    query: "John", 
    page: '1:1234',
    limit: 10,
    callback: function(result){
        
        // async
    }
});

Update item from an index

Index.update(id, string)

index.update(10025, "Road Runner");

Remove item from an index

Index.remove(id)

index.remove(10025);

Reset index

index.reset();

Destroy index

index.destroy();

Re-Initialize index

Index.init(\<options>)

Note: Re-initialization will also destroy the old index!

Initialize (with same options):

index.init();

Initialize with new options:

index.init({

    /* options */
});

Add custom matcher

BulkSearch.addMatcher({REGEX: REPLACE})

Add global matchers for all instances:

BulkSearch.addMatcher({

    'ä': 'a', // replaces all 'ä' to 'a'
    'ó': 'o',
    '[ûúù]': 'u' // replaces multiple
});

Add private matchers for a specific instance:

index.addMatcher({

    'ä': 'a', // replaces all 'ä' to 'a'
    'ó': 'o',
    '[ûúù]': 'u' // replaces multiple
});

Add custom encoder

Define a private custom encoder during creation/initialization:

var index = new BulkSearch({

    encode: function(str){
    
        // do something with str ...
        
        return str;
    }
});

Register a global encoder to be used by all instances

BulkSearch.register(name, encoder)

BulkSearch.register('whitespace', function(str){

    return str.replace(/ /g, '');
});

Use global encoders:

var index = new BulkSearch({ encode: 'whitespace' });

Call encoders directly

Private encoder:

var encoded = index.encode("sample text");

Global encoder:

var encoded = BulkSearch.encode("whitespace", "sample text");

Mixup/Extend multiple encoders

BulkSearch.register('mixed', function(str){
  
    str = this.encode("icase", str);  // built-in
    str = this.encode("whitespace", str); // custom
    
    return str;
});

BulkSearch.register('extended', function(str){
  
    str = this.encode("custom", str);
    
    // do something additional with str ...

    return str;
});

Get info

index.info();

Returns information about the index, e.g.:

{
    "bytes": 103600,
    "chunks": 9,
    "fragmentation": 0,
    "fragments": 0,
    "id": 0,
    "length": 7798,
    "matchers": 0,
    "size": 10000,
    "status": false
}

Note: When the fragmentation value is about 50% or higher, your should consider using cleanup().

Optimize / Cleanup index

Optimize an index will free all fragmented memory and also rebuilds the index by scoring.

index.optimize();

Pagination

Note: Pagination can simply reduce query time by a factor of 100.

Enable pagination on initialization:

var index = BulkSearch.create({ paging: true });

Perform query and pass a limit (items per page):

index.search("John", 10);

The response will include a pagination object like this:

{
    "current": "0:0",
    "prev": null,
    "next": "1:16322",
    "results": []
}

Explanation:

Perform query and pass a pointer to a specific page:

index.search("John", {
    
    page: "1:16322", // pointer
    limit: 10
});

Options

Phonetic Encoding

Compare Phonetic Search

Reference String: "Björn-Phillipp Mayer"

Memory Usage

Note: The data type of passed IDs has to be specified on creation. It is recommended to uses the most lowest possible data range here, e.g. use "short" when IDs are not higher than 65,535.

Author BulkSearch: Thomas Wilkerling License: Apache 2.0 License

@everything-registry/sub-chunk-1274 @infinitebrahmanuniverse/nolb-bul

6 years ago

6 years ago

6 years ago

6 years ago

6 years ago