Mongodb-watch-template NPM

mrex-template-service

Overview

Easily extendable template service that provides an ability to react to multiple MongoDB collection changes (Insert, Update document events). Users can extend a base class Task to implement custom handlers for the collection events without implementing any logic related to the database access.

Service comes with in-built lightweight Task scheduler and rate limiter, so developers don't have to worry about overloading the servers even in case of high event traffic.

Task Class

Task base class represents a handler that reacts to an event in particular MongoDB collection. Number of Tasks is not limited. Service can support multiple collections. Every subclass of Task should have a property called collection that should be the name of the MongoDB collection the Task is designed to react to.

Task has an optional property updateFilter. Task will only subscribe to the update events on the properties listed in it.

class ExampleTaskOne extends Task {
    public collection:string = config.example_one_collection
    public updateFilter: string[] = ["images", "details.address", "date"]

In this case Task will only receive update events if the images, date or details.address property gets changed. updateFilter supports dot notation for embedded documents and arrays. If updateFilter is omitted Task subscribes to all the updates to the collection.

Users should implement the updateEvent and insertEvent methods. Class will be instantiated only once, so it can hold a state between the events. Configured and instantiated database this.db is available to the users to access the mongodb database.

    async insertEvent(document:Document, id:ObjectId) {
        throw new Error('Not Implemented')
    }

    async updateEvent(updateDescription:any, id:ObjectId) {
        throw new Error('Not Implemented')
    }

insertEvent gets the newly created document as the document parameter and the id of this document.

updateEvent gets the information about the update event as the updateDescription parameter. ({ updatedFields: { <key>: <value> }, removedFields: [] }) and the id of the document.

In case there is a need to work with naked Mongodb events, handleEvent method can be overwritten. super(event) should be called in the overwritten method, otherwise insertEvent and updateEvent methods will no longer be called and it is fully the developers responsability to process the event.

    async handleEvent(event: any) {
    }

init method of a Task is called shortly after it is created, before the start of event processing. But init is never awaited, so in case of long-running task it might overlap with event processing.

API

execTasks(Task1, Task2, Task3, ...) - accepts Array of Task Classes. Initializes the tasks and starts watching corresponding Mongo Collections.

	execTasks([ExampleTask1, ExampleTask2, ExampleTask3])

Task - Base class for Tasks.

config - Configuration Object. All the paramteres in config.yaml will be accessable to use.

logger - Default logger.

createLogger(name: string) - Creates logger with specified name.

Logger - Logger type. Same as winston Logger.

configuration parameters

Service is configurable using the config.yml file or ENV variables. Any parameter defined in the config.yml can be substituted by the environmental variable of same name in uppercase. Configuartion data can be accessed through the config module. In config.yaml file is missing, service will exit with exit code 1.

project_name - defines the project name.
mongo_url - mongo connection string. Needs to include the DB name.
production - if set to true logger turns off the coloring feature. In the future will be extended to change behaviour of the service to suite the production environment.
management_collection - collection to store the management metadata.
resume_stream - if set to true, the service will continue processing mongoDB events from the last processed event.
health_endpoint - if set to true /health endpoint becomes available at 80 port locally to check service health.
server_port - port of the /health endpoint.
task_queue_length - determines the limit of the concurrent tasks processed by the service.

configuration options can be overwritten with environmental variables (all uppercase with same names: MONGO_URL, MANAGEMENT_COLLECTION, TASK_QUEUE_LENGTH, etc..)