Sc-dynamo-object-mapper NPM

TypeScript library starter

A starter project that makes creating a TypeScript library extremely easy.

DynamoDB Request abstraction

We provide an abstraction on top of the aws-sdk to execute requests. The fluent api should be self-explanatory always with the ability to access the underlaying plain request params to be very flexible. (If new api parameters are not implemented by the library yet, we can still use it).

Primary Key

To be clear about the used naming for keys, here is how we use it (same as in the official aws documentation):

DynamoDb has two key types Partition Key (hashkey - dynamodb internally uses a hash function to evenly distribute data items across partitions) and Sort Key (rangekey) The primary key can either be simple (only a partition key) or composite (combination of partition and sort key)

Condition Expression Builder

Expression Attribute Names

By default we create a substitution placeholder for all the attributes, just to not implement a blacklist with reserved words in the context of aws dynamodb.

attributename: age

attributeExpressionNames: {'#age': 'age'} attributeExpressionValues: {':age': {N: '10'}} expression: '#age = :age'

this works seemlesly for top level attribtues, but if we wanna build an expression for where the attribute needs to be accessed with a document path, we need some special logic attributeName: person.age

attributeExpressionNames: {'#person':'person', '#age': 'age'} attributeExpressionValues: {':age': {N: '10'}} expression: '#person.#age = :age'

we can't use #personAge: 'person.age' because if the dot is part of an attribute name it is not treated as metacharacter compared to when using directly in expression, so the above solution needs to be used

these are the accessor rules for nested attribute types n—for list elements . (dot)—for map elements

Object Mapper

Enum

For now i just implement a custom mapper for my enums, introduce new decorator @Enum

Null Values

Think about an attribute of type string where the value is an empty string, this is not a valid attributeValue to be persisted. There are two solutions for this problem. 1. Don't send the attribute to the backend 2. Use the NULL type to express the empty string

The default for now is to skip properties with non-valid values depending on type (string: empty string / set: empty set)

Notes

lodash with rollup and jest

We ended up using lodash-es to get it to work with rollup, there is some additional configuration required for jest Tree Shake we used the configuration linked (allowJs in tsconfig, and transform & transformIgnorePattern in jest config -> see package.json)

Decorators

Decorators are used to add some metadata to our model classes required by the mapper for some special cases.

This is an experimental feature and requires to set the

"experimentalDecorators": true
"emitDecoratorMetadata": true

typescript compiler options.

Additionally we rely on the reflect-metadata (https://www.npmjs.com/package/reflect-metadata) library for reflection api.

To get started with decorators just add a @Model() Decorator to any ts class. By default this enables the custom mapping functionality and will get you started to work with Dynamo DB and simple types (like String, Number, Boolean etc. but no custom classes for example)

We make heavy usage of compile time informations about our models and the property types. Here is a list of the types that can be retrieved from compile time information for the key design:type. (The metadata will only be added if at least one decorator is present on a property)

String Number Boolean Array (no generics) Custom Types

Map / Set will be Object

Generic information is never available due to some serialization limitations at the time of writing.

ES6 types like Set, Map will be mapped to Object when calling for the type via Reflect.get(design:type), so we need some extra info.

Collections

#Array Javascript Arrays with a a items of type String, Number or Binary will be mapped to a S(et) type, by default all other types are mapped to L(ist) type. If an item of an Array has a complex type the type can be defined using the @TypedArray() Decorator.

#Set es6 Set types will be marshalled to dynamoDb set type if the type of the set is supported, if the type is not supported it will be marshalled to an dynamoDB List.

When one of the following decorators is added, the value is marshalled to a List type. @SortedSet(), @TypedSet(complexType?)

##Model Here is the rule how a table name is built ${kebabCase(modelName)}s so for a model called Product the table will be named products, this is a default implementation. To Provide your own logic you can implement a TableNameResolver function and give it to the DynamoStore class when implementing a new instance.

Custom TableName @Model({tableName: tableName})

Types

Simple Type (no decorators requried to work)

String
Number
Boolean
Null
Array
Date (moment) is mapped by convention (see TODO:addLink Dates)

Complex Types (properties with these types need some decorators to work properly)

Set<simpleType | complexType>
Map
Array

Date

Two Date types are supported. Default JS Date and moment dates.

The type defines how a value will be mapped. Types can be defined using decorators (for complex types) or we use one of the following methods: fromDB -> use default for DynamoDB type (see type table) toDB -> use property value to resolve the type

design:type String, Number, Boolean, Undefined, Object

unsupported Set, Map, Date, moment.Moment

Dynamo DB

To map an js object into the attribute map required by dynamodb requests, we implement our very oppinionated custom mapper. We use the DynamoDB Document Mapper to map all «default» types to dynamodb attribute values.

There are some custom requirements for these cases:

MomentJs Dates
Use ES6 Map, Set types

Mapper Strategy:

-> To DB 1) check if we have some property metadata

  YES                                                      NO
  
  isCustomType                                             document client can map (check with typeof propertyValue for additional security)
  
  YES                 NO
  
  custom mapping      document client can map

-> From DB

Open Tasks

Null Values? How does DynamoDb treat empty lists, sets or emtpy strings?

Node Project Template

Usage

git clone https://github.com/alexjoverm/typescript-library-starter.git YOURFOLDERNAME
cd YOURFOLDERNAME

# Run npm install and write your library name when asked. That's all!
npm install

Start coding! package.json and entry files are already set up for you, so don't worry about linking to your main file, typings, etc. Just keep those files with the same names.

Features

Zero-setup. After running npm install things will be setup for you :wink:
RollupJS for multiple optimized bundles following the standard convention and Tree-shaking.
Tests, coverage and interactive watch mode using Jest
Prettier and TSLint for code formatting and consistency.
Docs automatic generation and deployment to gh-pages, using TypeDoc
Automatic types (*.d.ts) file generation
Travis integration and Coveralls report
(Optional) Automatic releases and changelog, using Semantic release, Commitizen, Conventional changelog and Husky (for the git hooks)

Excluding peerDependencies

On library development, one might want to set some peer dependencies, and thus remove those from the final bundle. You can see in Rollup docs how to do that.

The good news is here is setup for you, you only must include the dependency name in external property within rollup.config.js. For example, if you wanna exclude lodash, just write there external: ['lodash'].

NPM scripts

npm t: Run test suite
npm start: Runs npm run build in watch mode
npm run test:watch: Run test suite in interactive watch mode
npm run test:prod: Run linting and generate coverage
npm run build: Generage bundles and typings, create docs
npm run lint: Lints code
npm run commit: Commit using conventional commit style (husky will tell you to use it if you haven't :wink:)

Automatic releases

If you'd like to have automatic releases with Semantic Versioning, follow these simple steps.

Prerequisites: you need to create/login accounts and add your project to:

npm
Travis
Coveralls

Run the following command to prepare hooks and stuff:

npm run semantic-release-prepare

Follow the console instructions to install semantic release run it (answer NO to "Generate travis.yml").

Note: make sure you've setup repository.url in your package.json file

npm install -g semantic-release-cli
semantic-release setup
# IMPORTANT!! Answer NO to "Generate travis.yml" question. Is already prepared for you :P

From now on, you'll need to use npm run commit, which is a convenient way to create conventional commits.

Automatic releases are possible thanks to semantic release, which publishes your code automatically on github and npm, plus generates automatically a changelog. This setup is highly influenced by Kent C. Dodds course on egghead.io

Git Hooks

There is already set a precommit hook for formatting your code with Prettier :nail_care:

By default, there are 2 disabled git hooks. They're set up when you run the npm run semantic-release-prepare script. They make sure:

You follow a conventional commit message
Your build is not gonna fail in Travis (or your CI server), since it's runned locally before git push

This makes more sense in combination with automatic releases

FAQ

`Array.prototype.from`, `Promise`, `Map`... is undefined?

TypeScript or Babel only provides down-emits on syntactical features (class, let, async/away...), but not on functional features (Array.prototype.find, Set, Promise...), . For that, you need Polyfills, such as core-js or babel-polyfill (which extends core-js).

For a library, core-js plays very nicely, since you can import just the polyfills you need:

import "core-js/fn/array/find"
import "core-js/fn/string/includes"
import "core-js/fn/promise"
...

What is `npm install` doing the first time runned?

It runs the script tools/init which sets up everything for you. In short, it:

Configures RollupJS for the build, which creates the bundles.
Configures package.json (typings file, main file, etc)
Renames main src and test files

What if I don't want git-hooks, automatic releases or semantic-release?

Then you may want to:

Remove commitmsg, postinstall scripts from package.json. That will not use those git hooks to make sure you make a conventional commit
Remove npm run semantic-release from .travis.yml