datamatic v0.9.13

Datamatic

RxJS + JSON-Schema (Ajv) Based Observable and Validating Data Models and Pipelines

Online Developer Documentation

Goals

• Provide a means to quickly and easily validate complex data-sets
• Look and feel like a standard JS Object for ease of use and adaptability
• Automate data evaluation and transformation

Table of Contents

Installation Instructions

Usage Examples

• Basic Example
• Data Pipelines and Transformation

Developer Guide

• Model Class
  • Model Schemas Config
  • Model Proxy Object
  • model vs $model
• ItemsModel
• PropertiesModel
• BaseModel Class
• Pipeline Class

Installation Instructions

$ npm install datamatic

UMD Usage (React, Angular, Vue et al)

import * as datamatic from "datamatic";

CommonJS Usage for NodeJS

const {Model, Pipeline} = require("datamatic");

DOM Window Usage

    <script src="../../dist/datamatic.window.js"></script>
    <script language="JavaScript">
        const {Model, Pipeline} = window.datamatic;
    </script>

Usage Examples

Basic Example

The example below defines a Model that expects a name value and list of topScores items

const {Model} = require("datamatic");

// JSON-SCHEMA for Scores Collection
const schema = {
    "id": "root#",
    "type": "object",
    "properties": {
        "name": {
            "type": "string",
        },
        "topScores": {
            "type": "array",
            "minItems": 1,
            "maxItems": 3,
            "items": {
                "type": "object",
                "properties": {
                    "name": {
                        "type": "string"
                    },
                    "score": {
                        "type": "integer",
                        "default": 0
                    }
                },
                "required": ["name"]
            }
        }
    },
    required: ["name", "topScores"],
};


// instantiate our Model
const obj = new Model({schemas: [schema]});

// subscribes an observer to the Model
obj.subscribe({
    next: function (ref) {
        console.log("\t>> update succeeded!\n\t%s\n\t%s\n\n",
            "current object state:", "" + JSON.stringify(ref));
    },
    complete: function (ref) {
        console.log("\t>> %s",
            "object is frozen and no longer editable");
    },
    error: function (e) {
        console.log("\t>> update FAILED with error:\n\t%s\n",
            JSON.stringify(e));
        console.log("\tcurrent object state:\n\t%s\n", obj);
    },
});

// populate the Model with data
// -- this will trigger the "next" notification
obj.model = {
    name: "JSONville",
    topScores: [{
        name: "Player 1",
        score: 12300000,
    }, {
        name: "Player 2",
        score: 45600000,
    }, {
        name: "Player 3",
        score: 78900000,
    }]
};

// update the model
// this will trigger the next notification
obj.model.topScores[0].score++;

// invalid attempt update the model
// this will trigger the error notification
// reason: "topScores/items/score" is type is integer 
obj.model.topScores[0].score = "1234";

// invalid attempt update the model
// this will trigger the error notification
// reason: "topScores" is marked as required
delete obj.model.topScores;

Refer to the examples demo in ./examples/basic-usage for more usage examples

Data Pipelines and Transformation

const {Pipeline} = require("datamatic");

/*
    defines a schema that requires name, age and active attributes
    filters out all items that do not conform to JSON-Schema below
 */
const schema = {
    type: "object",
    required: ["name", "age", "active"],
    properties: {
        name: {
            $comment: "names must be in form: First Middle|MI Last",
            type: "string",
            pattern: "^[a-zA-Z]{1,24}\\s?[a-zA-Z]?\\s+[a-zA-Z]{1,24}$",
        },
        age: {
            $comment: "age must be a number equal to or higher than 21 and lower than 100",
            type: "number",
            minimum: 21,
            maximum: 100,
        },
        active: {
            $comment: "active must equal true",
            type: "boolean",
            const: true,
        },
    },
};


const pipeline = new Pipeline(
    [
        // By nesting an item schema within an iterator, the schema is applied as a filter
        schema,
        // the list can go on ...
    ],
    // the list can go on ...
);

pipeline.subscribe({
    // should only contain active people who are 21 and over and name pattern match
    next: (d) => console.log(`\nfiltered results:\n${JSON.stringify(d)}`),
    // it should not encounter an error unless it is critical, so full stop
    error: (e) => console.error(`\ngot error:\n${JSON.stringify(e)}`),
});

pipeline.write([
    {name: "Alice Dodson", age: 30, active: false}, // will be filtered because of active status
    {name: "Jim-Bob", age: 21, active: true}, // will be filtered because of name format
    {name: "Bob Roberts", age: 38, active: true}, // will pass
    {name: "Chris Appleton", age: 19, active: true}, // will be filtered because of age
    {name: "Fred Franks", age: 20, active: true}, // will be filtered because of age
    {name: "Sam Smith", age: 25, active: true}, // will pass
    {name: "", active: null}, // will be filtered because of invalid object format
]);

Developer Guide

Model Class

This class represents the Document entry point

Model Schemas Config

Model Proxy Object

This is the Data Model most usage will be based around. It is a Proxy Object that has traps for all operations that alter the state of the given Array or Object

model vs $model

In usage, model always references the Proxied Data Model for validation and operation where $model references the owner Model Class

example:

 const _owner = new Model({schemas: [schema]});
 
 // access the root model:
 console.log(`JSON.stringify(_owner.model)`);
 
 // access the model's owner Model Class:
 const owner = _owner.model.$model;
 console.log(`is frozen: ${owner.isFrozen}`);
 
 // call toString on Owner
 console.log(`stringified: ${owner}`);
 
 // obtain model from  it's Owner
 console.log(`stringified: ${JSON.stringify(owner.model)}`);
 

ItemsModel

subclass of Model Class

Represents an Items (Array) entry in the given schema Note: the model param presents a Proxied Array, with all Array.prototype methods trapped and validatable

PropertiesModel

subclass of Model Class

Represents an Properties (Object} entry in the given schema

BaseModel Class

Pipeline Class