Sengi-docstore-dynamodb NPM

Sengi DocStore DynamoDB

This package is part of the Sengi family.

A wrapper for Amazon's AWS Dynamo DB that implements the Sengi document store interface.

Installation

npm install sengi-docstore-dynamodb

Usage

The DynamoDbDocStore implements the DocStore interface defined in sengi-interfaces.

To instantiate a DynamoDbDocStore you have to provide the following parameters:

dynamoUrl - A url that identifies your Dynamo DB instance. This should be the phrase 'production' or the url to a local instance.
region - The AWS region where the Dynamo DB resides. If using a local instance this value is not used.
generateDocVersionFunc - A function () => string that returns a string of random characters.
getTableNameFunc - A function (docTypeName: string, docTypePluralName: string, options: DocStoreOptions) => string that returns the name of the table.

const dynamoDbDocStore = new DynamoDbDocStore({
    dynamoUrl: 'production',
    region: 'us-east-1',
    generateDocVersionFunc: () => crypto.randomBytes(Math.ceil(10)).toString('hex').slice(0, 20),
    getTableNameFunc: (docTableName, docTypePluralName) => `mySystem.myEnv.${docTypePluralName}`
  })

This example uses the standard NodeJs crypto library to produce a string of 20 random hex characters for generateDocVersionFunc.

Filters

Filter expressions are expected to be a DynamoDbFilterExpression object. This interface is exported from the library.

This is an example filter expression returned from a DocType filter implementation:

const filterExpression = {
  indexName: 'mySecondaryIndex',
  condition: 'docType = :docType and heightInCms > :heightParam', // can use just the equality, or equality and sort
  conditionParams: {
    ':docType': 'tree',
    ':heightParam': 200
  }
}

DynamoDB defines a long list of reserved words. If the field of a doc uses one of these names it is necessary to alias the field (# prefix) using a condition param name as shown below:

const filterExpression = {
  indexName: 'mySecondaryIndex',
  condition: '#docTypeVar = :docType and heightInCms > :heightParam',
  conditionParams: {
    ':docType': 'tree',
    ':heightParam': 200
  },
  conditionParamNames: {
    '#docTypeVar': 'docType'
  }
}

Indexes

This provider requires each table to be partitioned on a string field named id. To do this, specify the key schema as follows:

{
  "TableName": "myTable",
  "KeySchema": [{ "AttributeName": "id", "KeyType": "HASH" }]
}

DynamoDB can create secondary indexes but these are not quite analogous to secondary indexes on Mongo, Cosmos or RDBMS systems. In DynamoDB a secondary index is essentially a copy of (or a subset of) the original table with a different partition (hash) and sort key.

For authoratitive child documents, you will need a mechanism for finding all the child records that are owned by a given parent. You do this by filtering the child table based on the parent table id. To enable this, create a global secondary index specifying the parent record's id as the hash code. This will allow you to create a DocType filter on the child table that targets the parent id.

For warehousing documents, you can specify any field to act as a partition key. If there is no sensible field to use, perhaps because you want to filter across the whole dataset, then you can use the docType field as the partition key. In this scenario you'll only have 1 partition so it's important not to project too many fields into it.

Limitations

DynamoDB does not support the concept of an arbitrary numerical offset when fetching documents, so the offset parameter in queries will be ignored.

If you request a field that has not been projected into a secondary index, Sengi will not know. You will simply not receive that field.

Setup

To run the tests you will need a locally running instance of the Dynamo database with the test tables. The following commands setup a local copy on docker and creates the necessary tables:

docker run -d -p 8000:8000 amazon/dynamodb-local:1.13.4
npm run setup

To instantiate a connection with a production DynamoDB instance you will need to install the aws cli tool. You then call aws configure to establish credentials (AWS Access Key ID, AWS Secret Access Key, and Default Region Name) that can invoke operations on DynamoDB.