Rest-api-starter NPM

rest-api-starter

REST API starter is a small tool kit that will help you writing simple REST APIs in Node.js using MongoDB.

It will not cover every use case in the world, but will be useful for most. It is oriented to a micro-service architecture, so it provides some features that will make it easier to administer your services.

Installing

It was tested for Node.js 4+ and 6+.

npm install --save rest-api-starter

This library wraps Express.js, Bluebird, Winston and Config, which are core to its features and by using them you will get consistency across all your applications.

The config and bluebird libraries are exposed as:

Bluebird: require('rest-api-starter').Promise
Config: require('rest-api-starter').configuration

Config

The toolkit uses the library config to manage configuration. You can import it yourself and use it without adding the dependency to the package.json file, as we exposed it so you can use the same configuration the toolkit does.

const configuration = require('rest-api-starter').configuration;

You can find, in this repository, in the file config/default.json, an example of the configuration that is mandatory for this toolkit to work. The same example is written below:

{
  "app": {
    "http": {
      "port": 8100,
      "host": "0.0.0.0",
      "queue": 10,
      "secret": "",
      "transactionHeader": "X-REST-TRANSACTION"
    },
    "log": {
      "level": "info",
      "transports": [
        {
          "type": "console"
        },
        {
          "type": "file",
          "filename": "/tmp/rest-api.log"
        }
      ]
    },
    "mongo": {
      "url": "mongodb://localhost:27017/rest-api"
    }
  }
}

Local store

The localStore is another tool present on the toolkit, that you can use to store values that you need available for a transaction.

When you call localStore.create() you are creating a store for the current transaction (e.g. transaction = http request), it receives a function that will be called after the store is created, so from that point on you can start writing and reading to it.

You can use localStore.putValue() to store values available for the current transaction, giving it a key, value pair, and localStore.getValue() to retrieve a value present in the current transaction, giving it the key.

Logger

This toolkit gives you a configurable Winston logger wrapper, which you can use by executing require('rest-api-starter').logger(), passing the name of the logger you want to create.

You can configure the logging transports via the config module, below is an example config/default.json.

{
    "app": {
        "log": {
            "level": "info",
            "transports": [
                {
                    "type": "console"
                },
                {
                    "type": "file",
                    "filename": "/tmp/rest-api.log"
                }
            ]
        }
    }
}

The above snippet configures two transports:

A console transport.
A file transport that appends the log lines to a file in /tmp/rest-api.log.

The logger you import by creating one using the library isn't the actual winston logger, but a wrapper of it that exposes just the essential methods for logging in the different levels: debug, info, warn (and the alias warning), error and the is<level>Enabled functions.

The log format is not yet configurable, so it will look like this:

2017-01-13T22:29:50.842Z [INFO] - [tid: <some-transaction-identifier>] - [server-initialization] Received request GET at /bad-request-endpoint

ISO Date
Log Level
The transaction Id that was stored in the local store by the logging filter (more info below)
Logger name
Message

The transaction identifier provides a way of knowing which lines were generated by what http request. So, basically, you have a way of knowing which lines of log belong together.

DB

The db tool (you can use by doing require('rest-api-starter').db) is a wrapper of mongodb, which provides some utility functions:

all(collectionName, sortOptions): Retrieves all documents in the provided collection. The sort options works as described in mongodb docs (https://mongodb.github.io/node-mongodb-native/markdown-docs/queries.html#sorting). Resolved with the array of objects.
find(query, collectionName, sortOptions): Retrieves all documents matching the given query in the provided collection. Resolves with the array of objects.
findById(id, collectionName): Executes find with the query {"id": id} and the provided collection. Resolved with the found object.
del(id, collectionName): Removes a document from the provided collection with the query {"id": id}. Resolved without parameters.
insert(document, collectionName): Inserts the given document in the provided collection. Resolves with the inserted document.
update(id, document, collectionName): Updated the document matching the query {"id": id} with the given document in the provided collection. Resolves with the updated document.

All this functions return bluebird's promises, and on any error, this promises will reject.

This tool expects the next configuration to be present:

app.mongo.url=mongodb://user:password@host:port/db

Validator

The validator tool provides a validator builder based on JSON schema. The builder is invokes with two parameters:

schema: A JSON Schema compliant JSON that describes the structure to be validated.
customValidations: A function that applies custom validations that can not be specified with just the schema. This is an optional parameter.

The build object will expose one function, validate, which receives two parameters: object, and id.

The object will be validated using the provided schema, and, if the id parameter was provided, the validator will check that the object has an id key and it matches the provided id.

Then, if provided, customValidations will be invoked with four arguments:

schema: The provided JSON Schema.
object: The object that needs validation.
id: The provided id. undefined if it isn't present.
errors: The array of resulting errors where you should push the errors of your custom validations.
finishValidation: This is a function that you must call after you've done your custom validations. This was created to support asynchronous validations. If you don't execute this function (finishValidation()), the promise created by the validator will never resolve nor reject.

The validator returns a promise, that will resolve with the given object if validation succeeded, and will reject with the response described below if not.

{
    'name': 'validation.error',
    'errors': [
        'schema.user.instance.name',
        'schema.user.instance.password'
    ]
}

The our of the box error handlers will automatically handle this rejection with a Bad Request (400) response. More information below.

Starting a REST API

You just need to import the toolkit server builder, and give it a router. The router is a function that receives an express application, and adds the routes to it.

Of course, in that function you can do anything you want to the application, as it is just an express application, not wrapped, not monkey patched, just the object that is returned by doing new Express();.

const serverBuilder = require('rest-api-starter').server;

const mySimpleRouter = (application) => {
    application.get('/endpoint', (request, response, next) => {});
};

const server = serverBuilder(mySimpleRouter);

Notice that serverBuilder receives 3 parameters:

router (mandatory): Implementation of your routes and custom server configuration.
customNotFoundHandler (optional): Will override default 404 handler.
customErrorHandler (optional): Will override default error handler.

Before Routing

Before the router is executed, the starter will configure a request logging filter (more on it below), compression, cookie-parser, and body-parser for JSON.

Logging Filter

The logging filter that's configured looks like this:

app.use(function(request, response, next) {
    localStore.create(() => {
        const transactionId = request.get(transactionHeaderName) || uuid.v4().substr(0, 7);
        response.append(transactionHeaderName, transactionId);
        
        localStore.putValue('req', request);
        localStore.putValue('res', response);
        localStore.putValue('tid', transactionId);
        
        logger.info(`Received request ${request.method} at ${request.url}`);
        next();
    });
});

By using the localStore, the filter will retrieve a transaction identifier from the request headers (or generate one if none is found) and store it with the key tid. It will also add the transaction id as a header in the response.

The name of the header from which the filter will extract the transaction identifier, can be configured via the key app.http.transactionHeader.

The filter will add the request and response too to the local store, and then log the method and URL using the toolkit logger.

Express filters

After the logging filter is configured, compression, cookie parser and body parser will be configured as follows:

app.use(compression());
app.use(cookieParser(configuration.get('app.http.secret')));
app.use(bodyParser.json());

As you can see, out of the box you have gzip compression, json support, and a secured cookie parser with the key you configure through the key app.http.secred.

After this, your router will be executed passing the application to it.

Each of these middlewares can be disabled by a configuration:

app.http.compression: If not set, or set to true, compression middleware will be configured. If set to false compression middleware configuration will be skipped.
app.http.cookies: If not set, or set to true, cookie parser middleware will be configured. If set to false cookie parser middleware configuration will be skipped.
app.http.json: If not set, or set to true, json body parser will be configured. If set to false, no body parser will be configured by default.

After Routing

Error Handlers

The server builder will handle errors using Express filters as per Express' documentation recommendations. Will handle everything passed to the next function and check for a couple conditions described below to choose the status code.

The response body in a case of error is defined by the next JSON schema:

{
    "status": {
        "type": "number",
        "description": "Response http status code."
    },
    "message": {
        "type": "string", 
        "description": "A short description of the ocurred error."
    },
    "detail": {
        "description": "This will contain detailed information about the error. The type of this field will vary according to the type of the error."
    }

For the out of the box error handling to work, your code will need to comply with a couple standards.

Validation Errors (400)

Express' next function shall be called with a JSON containing the key name with the value validation.error and a key errors with an array of errors. Check the validator tool, it complies with this standard. Example:

{
    "name": "validation.error",
    "errors": [ // List of validation errors. 
        "schema.user.name",
        "schema.user.password"
    ]
}

If the above JSON is passed to Express' next function, the name will be assigned to the message key of the error response, and the array of errors to the detail. This results in a response body as follows:

{
    "status": 400, 
    "message": "validation.error", 
    "detail": [ 
        "schema.user.name",
        "schema.user.password"
    ]
}

Not Found Errors (404)

Out of the box, there are two cases when this error will present itself:

When a client request a resource that doesn't exist (URL/HTTP VERB combination).
When a query by id is executed in the db tool, and no document is returned.

You should, any way, throw this error whenever you query for a unique document and you don't find it.

The response in the case of an invalid URL/HTTP verb combination looks as follows:

{
    'status': 404,
    'message': 'url/method combination not found',
    'detail': {
        'url': req.url,
        'method': req.method
    }
}

For the case of a document not found, you need to pass a JSON object with the key name with the value document.not.found, the key message and detail will be used to build the response. Below is an example of what you should send:

{
    'name': 'document.not.found',
    'message': 'Query for unique document returned empty.',
    'detail': { 
        'collection': collectionName,
        'id': id
    }
}

Remember that the message and detail field are free, you can fill them as you need. The above object will result in the next response body:

{
    'status': 404,
    'message': 'Query for unique document returned empty.',
    'detail': { 
          'collection': collectionName,
          'id': id
      }
}

If customNotFoundHandler was provided, this whole process will be overwritten by it.

Unhandled Errors (500)

Any other errors that are not compliant with this standards will be handled doing a .toString(). The response will look as follows:

{
    'status': 500,
    'message': 'unhandled error',
    'detail': err.toString()
}

This is the case of a throw new Error("this shouldn't happen");.

If customErrorHandler was provided, this whole process will be overwritten by it.

Server Start Up

After the error handlers are in place, the server will start using the next configuration keys:

app.http.host: Host to bind the server to.
app.http.port: Port to bind the server to.
app.http.queue: Maximum length of the queue of pending connections.

If you configured everything, you now have a node.js REST API up and running.

Client

The client tool, which you can import by doing require('rest-api-starter').client, is a wrapper of superagent-promise using bluebird.

It's a builder that receives two parameters:

The endpoint. E.g. https://some-api.domain.com:8080.
A global timeout (number, milliseconds).

The builder returns an object with four functions:

get(uri, headers, requestTimeout): Executes a GET request to the URL built by doing ${endpoint}${uri}, with the given headers. It returns a promise that resolves with the response body as a JSON if the request was successful, and rejects with the whole error if not.
post(body, uri, headers, requestTimeout): Executes a POST request to the URL built by doing ${endpoint}${uri}, with the given body and headers. It returns a promise that resolves with the response body as a JSON if the request was successful, and rejects with the whole error if not.
put(body, uri, headers, requestTimeout): Executes a PUT request to the URL built by doing ${endpoint}${uri}, with the given body and headers. It returns a promise that resolves with the response body as a JSON if the request was successful, and rejects with the whole error if not.
del(uri, headers, requestTimeout): Executes a DELETE request to the URL built by doing ${endpoint}${uri}, with the given headers. It returns a promise that resolves with the response body as a JSON if the request was successful, and rejects with the whole error if not.

If provided, the client will apply the given global timeout to every request made by this instance. If a request time out (the third parameter) is provided for any of the four methods (get, post, put, del), that time out will be applied over the global one. Basically, specific timeout wins over global timeout.

The timeouts will be applied as a deadline for the whole request. See superagent documentation about timeout.

This client adds the transaction identifier, stored by the logging filter in the local store, to every request in the configured header (app.http.transactionHeader), so if you use it outside an HTTP request, keep in mind that it expects a value will be present for the key tid in the localStore tool for the current transaction.