simple-search-se v1.0.1

Instructions for use

Installation

simple-search can be installed in two ways:

NPM: simple-search can be installed through NPM if the source code is not required. This is shown below:

npm -i simple-search-se

Source code: if you want to edit the system you can clone it from https://gitlab.com/Mark-Harris77/simple-search to compile the source code run the following commands:

npm install
npm run-script build

Sample Projects

Included in the source code are a few example projects to showcase functionality, these are written in JavaScript and can be executed using the following command:

node "path to JavaScript file"

Plese note that TensorFlow.js was excluded from the dependencies of the project as it drastically increased the bundle size, however it can be installed separately using the following command:

npm install @tensorflow/tfjs @tensorflow-models/universal-sentence-encoder

Documentation

search-config: This file will be created automatically if it does not exist, simply run the search engine to create it. The options for the config are the following:

index: inMemoryIndex, clusteredIndex
parser: JSON, text, tweet
retrievalModel: BM25

Usage: some commands differ depending on using the source code or NPM versions

To import the project:

source code:

const { SimpleSearch } = require("<path/to/dist/app/simpleSearch>")

NPM:

const simpleSearch = require("simple-search-se")

Initializing the search engine:

source code:

let se = new SimpleSearch()

NPM:

let se = new simpleSearch.SimpleSearch()

Add a file or directory to the index:

file: se.addFile("<path to file")
directory: se.addDir("path to directory")

Preform a query:

let results = await se.search("query")

Evaluate System using trec_eval:

se.evaluate("path to query file")

Methods

Below are the methods exposed to the user

/**
* Function to add a file to the index
* @param filepath - path the the file to be indexed
*/
se.addFile(filepath: string)

/**
* Async function to add the entire contents of a directory to the index
* @param dirpath - path to the directory to add
*/
se.addDir(dirpath: string)

/**
* Preform trec_eval evaluations, writing output to stdout
* @param queryPath - path the the file containign queries
*/
se.evaluate(queryPath: string)

/**
* saves the index to a file located at "./search-tmp/data/index/index.json"
*/
se.save()

/**
* Function to load a previously created index from a file
* @param filepath - path to the index.json file to be loaded
*/
se.load(filepath: string)