Es-odm NPM | npm.io

ElasticSearch 8 Node.js ODM library

Basic information

This library maps ES documents and functions to JS classes

Alias

ODM models are represented via ES aliases
Calling functions like createIndex() will create an underlying index and also sets an alias into it
Most of the communication is accomplished via aliases, real indices are used only when necessary
- This is to simplify some processes like reindexing/cloning/migrating from source (old) index into the new one
  - Thanks to this, we can create new index, migrate data from the original one, than switch the aliases and delete the original one.
  - And the index are still in human-readable format
Alias consist from two parts:
- tenant (AKA prefix) - defaults to *
- name - main index name
Final alias looks like this:
- tenant_name

Work with aliases

Alias name is specified when model is created, like this - createClass('myIndex')
Tenant can be specified using createClass('myIndex', void 0, 'myTenant')
- or later by using .in('myTenant')
Function in('tenant') (and immediateRefresh(bool) as well) creates new ODM model with updated values, original ODM remains unchanged
example:
- createClass('myIndex').in('default') -> default_myIndex
- createClass('myIndex') -> *_myIndex
- createClass('myIndex', void 0, 'tenant') -> tenant_myIndex
- createClass('myIndex', void 0, 'tenant').in('changedTenant') -> changedTenant_myIndex
When creating a new instance (or performing some operations), the alias cannot contain any wildcard
- it means you must fully specify the tenant

Underlying index

Underlying index is in a very similar format, just also contains unique identifier:
- tenant_name-id
This library is made to use aliases, but technically speaking it should be also working with ES indices directly (or even with mixture of indices and aliases)
- It is necessary to avoid any conflicts between indices / aliases
- Newly created indices will always be created as aliases with underlying indices

Usage

const { createClass, BulkArray, BaseModel, JointModel, esClient, setClient, esErrors, setLoggerConfig, setLoggerUidFunction } = require('odm');
createClass function creates new class
- Each class contains several properties
  - schema, alias, _tenant, _name and _immediateRefresh
- Class static functions profit from those properties
  - e.g. static search() uses them to get alias
  - You may write your own static function, which will use those properties
    - Or rewrite existing functions
      - Which may influence another functions
    - Or redefine class properties
- Instances have access to those properties / functions
  - myInstance.constructor.alias
  - And they use them during ES operations
    - eq. to validate data, ...
- Whenever we need new / independent class, we must either create a new one or clone an existing one
Additionally, library provides BulkArray
- Class inherited from Array
- Contains methods which uses ES Bulk API
  - save, delete and reload
  - Plus some additional methods to dynamically mark and remove ODM instance - see in code
Exported BaseModel should be used mainly for instanceof checks
- Do not create instances from it / do not change its parameters (unless you really know what you are doing)
JointModel is a special class used to "record" searches from multiple distinct ODM classes and then perform single search with different queries above multiple indices
setClient replaces ES client singleton
- Should be called once at application startup
- New client is then used - even in already created classes
esErrors - Errors from underlying ES library
setLoggerConfig - Replaces default logger configuration
setLoggerUidFunction - Pass custom function to be used to generate ID, which is passed to generated logs

Class usage

Create new class

To create new class, call createClass(name, ?schema, ?tenant):
- name - essential, it must be specified
- schema - may be undefined or Joi object, it is there, so we can do additional validations
- tenant - Serves as part of the index (prefix)
  - This is required and cannot contain underscores (_)
  - By default, it is *
    - You can rewrite it later by using .in('newTenant')
      - It will create inherited class
      - const SpecificTenantClass = MyClass.in('newTenant');
    - You must specify it before creating new instance
    - When searching for records, found records will have correctly set tenant

Modify / clone existing class

Class can be modified / cloned using:
- in(tenant)
  - Returns class clone with tenant changed to given value
- immediateRefresh(bool)
  - Returns class clone with immediate refresh set to given value
    - By default, new classes (and its instances) does perform refresh: true with all write operations -> You can use this to disable it
- clone(?changes)
  - Clones class
  - changes is optional object to set cloned class properties
  - This method is internally used by in() and immediateRefresh()
  - You should not need to use it
Cloning class means:
- New inherited class is created
- Changes made afterwards to cloned class are NOT transferred to original one

Instances

Instances are made from prepared class
- Manually:
  - You prepare class and then you can call new MyClass(?data, ?_id, ?_version, ?_highlight, ?_primary_term, ?_seq_no, ?_score, ?_sort)
    - data is optional object whose properties are placed to instance
    - _id is optional ES _id
    - _version is optional ES _version
    - _highlight is optional ES highlight
    - _primary_term is optional ES _primary_term
    - _seq_no is optional ES _seq_no
    - _score is optional ES _score
    - _sort is optional ES sort
- From static functions:
  - When you call functions like findAll(), search(), ...
    - Instance is made from class and correct data is loaded from ES
Instance contains only ES data and methods to save / reload / validate / ...
- All ES properties, search functions, ... are saved in class
NOTE: _id, _version, _highlight, _primary_term, _seq_no, _score and _sort are not enumerable

BulkArray

For ES Bulk API, you can use BulkArray
Class inherited from Array
All static search functions in BaseModel class returns BulkArray of instances instead of Array
Provides bulk functions:
- async save(?useVersion)
  - Saves all items to ES
    - Not existing ids are generated and pushed to instances
  - Returns ES response
  - useVersion - Checks if version match
    - uses sequence numbers internally, if not presented, it will fetch them and checks version automatically
- async delete(?useVersion)
  - Deletes all items from ES
  - Returns ES response
  - useVersion - Checks if version match - uses sequence numbers internally, if not presented, it will fetch them and checks version automatically
- async reload()
  - Reloads all instances in bulk array

Examples:

class Counter extends createClass('counter', Joi.object()) {
  async customFunction() {
      return 666;
  }
  
  static customStatic() {
      return new this({ a: 4 }, `myId`);
  }
  
  static async search(...args) {
      // Rewritting static function -> affects search, find and findAll
      const results = await super.search(...args);
      if (results.length <= 0) {
          throw Error(`Nothing`);
      } else {
          return results;
      }
  }
}

...

const MyCounter = Counter.in('default'); //Index is 'default_counter'

const allCounters = await MyCounter.findAll();    //BulkArray with all counters in index
await allCounters.delete();   //Everything deleted from ES

const instanceCounter = new MyCounter({ counter: 15 }, 'myId');   //Prepared new instance
await instanceCounter.save(); //New instance saved

Functions / Methods

Internal getters

static get alias
- Returns class full alias, usable for ES queries
- It consists of tenant and name

Class level API

static async search(body, ?from, ?size, ?additional)
- Performs ES search
- Returns BulkArray of instances
- Used by another static functions
  - Redefining this function will affect their behavior
- User must specify body
  - alias is already defined in the class
- from and size are optional
  - Returns all results if from / size are not specified, no matter how many there are
- additional is an optional object with additional data

static async *bulkIterator(body, ?source, ?bulkSize, ?additional)
- Async generator for iterating over bulks of documents
- body, source and additional arguments are the same as in the search function
- bulkSize is optional custom size of a bulk
static async *itemIterator(body, ?source, ?bulkSize, ?additional)
- Async generator for iterating over documents
- Arguments are the same as in the bulkIterator generator
static async findAll()
- Finds all entries in ES, matching class alias
- Uses this.search()
  - Returns BulkArray
static async find(ids)
- Performs ES 'search' query
- Always returns BulkArray of instances
- Uses this.search()
static async get(ids)
- Performs ES 'get'
- If 'ids' is strings, returns single instance
- Else if 'ids' is an array of strings, returns BulkArray of instances
static async head(ids)
- Performs ES 'mget' without '_source' field
- Throws when any of the IDs is not found
- If 'ids' is strings, returns single object
- Else if 'ids' is an array of strings, returns array of objects
static async delete(ids, ?version)
- Performs ES 'delete'
- Uses bulk API
- Class alias must be fully specified
- Returns ES response
- If version is specified, only one id is allowed
static async exists(ids)
- Performs ES 'exists'
- If 'ids' is strings, returns single boolean
- Else if 'ids' is array of strings, returns array of booleans
  - true if exists, else otherwise
static async update(ids, body)
- Performs ES 'update'
- Uses bulk API
- Returns ES response
static async count(?body)
- Performs ES 'count'
- Returns number
static async updateByQuery(body)
- Performs ES 'update_by_query'
- Returns ES response
static async deleteByQuery(body)
- Performs ES 'delete_by_query'
- Returns ES response

Indices API

static async createIndex(?body, ?setAlias = true)
- Creates index given by current class
- body is optional settings
- setAlias is used when we do not want automatically set an alias to newly create index
static async getIndex()
- Returns ES index of ODM alias
- Returns string
static async aliasIndex(index)
- Puts alias of current ODM onto selected index
static async deleteAlias()
- Deletes ODM alias from ES, underlying index remains unchanged
static async aliasExists()
- Checks ODM alias existence
- Returns boolean
static async indexExists()
- Checks ODM index existence
- Returns boolean
static async deleteIndex()
- Deletes alias and index given by current class
static async getMapping()
- Gets mapping of index given by current class
static async putMapping(mapping)
- Puts mapping to index given by current class
static async getSettings(?includeDefaults = false)
- Gets settings of index given by current class
static async putSettings(settings)
- Puts settings to index given by current class
static async reindex(destinationModel, ?script)
- Reindex from current model class to selected one
- Destination may be chosen by ODM or by alias/index string
- script specifies optional painless script to be used
static async cloneIndex(?settings)
- Creates clone of ODM index
- Current index has to be manually blocked for write (be made read-only)
- settings is optional settings to be used for newly created index
- Returns newly created index
static async refresh()
- Refreshed current index / indices
static async openPIT()
- Opens Point in Time in given index
static async closePIT(id)
- Closes given Point in Time

Instance level API

async save(?useVersion)
- saves or re-saves instance
- it uses specified _id or generates new one if not specified
  - it uses ES 'index' function
- useVersion - checks if version matches, _id and _version must be specified
  - sequence number will be fetched automatically in not presented
async reload()
- Reloads instance data from ES
- Uses get and new this.constructor() internally
async delete(?useVersion)
- Deletes an instance from ES
- _id must be specified and entry must exist in ES
- useVersion - checks if version matches, _version must be specified
  - sequence number will be fetched automatically in not presented
clone(?_preserveAttributes)
- Returns clone of current instance
  - Deep copy
- Cloned instance is created from the same class
- preserveAttributes (defaults true) can be used to preserve attributes (_id, _version, ...)
async validate()
- Validates instance using Joi
- Throws in case of error / incorrect data

Class copy

static clone(?changes)
- Creates class copy
- In fact, it creates new class which inherits from the current one
- changes is an optional object with properties to be set
static in(newTenant)
- Clones class using clone()
- Sets given tenant
static immediateRefresh(newRefresh)
- Clones class using clone()
- Sets given refresh mode

Other

static _parseIndex(index)
- Parses given index (or even alias) and returns its parts
static __checkIfFullySpecified(functionName)
- Checks if current class is fully specified, meaning it doesn't contain any wildcard in alias
- Throws otherwise
static __getConstructor(searchResult, constructorCache)
- Returns the correct class (constructor) for given search result
static _alterSearch(body)
- Alters the search function query (and also query of other functions that use the query)
- By default, just passes the body without any change, but can be rewritten
static _unpackData(source)
- Alters the fetched ES documents before the instance is created from them
- By default, just passes the source without any change, but can be rewritten
async _packData(cache)
- Alters the instance data before it is saved to ES
- By default, just passes the instance data without any change, but can be rewritten
static async _afterSearch(instances, ?cache)
- Special function called for each record founded by search / find / findAll / get
- By default, it is empty, but you can overwrite it in your code
static async _getBulkSize()
- Returns optimal size of bulk for given model
- Used in search function for pagination