node-server-engine v1.2.9

Node Server Engine

Framework used to develop Node backend services. This package ships with a lot of features to standardize the creation of services, letting you focus on the business logic.

• Node Server Engine
  • Install
  • Entities
    • Server
      • Server Configuration
    • Endpoint
      • Endpoint Configuration
      • Methods
      • Authentication
      • File Upload Middleware
    • Socket Client
      • Socket Client Options
      • Properties
    • Message Handler
    • Redis
    • Sequelize
    • Pub/Sub
    • PushNotification
    • Localizator
    • ElasticSearch
    • Translation Manager
    • Error Reporting
      • Common options
      • Severity
        • Log Levels
      • EngineError
      • WebError
  • Middlewares
    • Swagger Docs
      • Structuring Documentation
      • Schemas and Responses
    • User Resolver
    • Gemini File Upload
    • Multipart File Uploader
    • checkPermission
  • Utilities
    • Request
    • TLS Request
    • TLS Config
    • Send Push Notification
    • Send Email
    • Gemini File Upload
    • Filter
    • Database Migration
    • Environment Variables Verification
      • Assertions Available
      • Wait

Install

To start a new service, it is highly recomended that you clone it from our template. It will already include all the necessary tools and boilerplate.

If you need to install it manually:

npm i node-server-engine

Entities

Server

The server class encapsulates all of the express boilerplate. Instantiate one per service and initialize it to get started.

import { Server } from 'node-server-engine';
const server = new Server(config);
server.init();

Server Configuration

Endpoint

Endpoint encapsulates the logic of a single endpoint in a standard way.

The main function of the endpoint is called the handler. This function should only handle pure business logic for a given endpoint.

An endpoint usually has a validator. A validator is a schema that will be compared to the incoming request. The request will be denied if it contains illegal or malformed arguments. For more detail see the documentation of the underlying package express-validator.

import { Endpoint, EndpointMethod } from 'node-server-engine';

// A basic handler that returns an HTTP status code of 200 to the client
function handler(req, res) {
  res.sendStatus(200);
}

// The request must contain `id` as a query string and it must be a UUID V4
const validator = {
  id: {
    in: 'query',
    isUUID: {
      options: 4
    }
  }
};

// This endpoint can be passed to the Server
new Endpoint({
  path: '/demo',
  method: EndpointMethod.GET,
  handler,
  validator
});

Endpoint Configuration

new Endpoint(config)

const addNodeEndpoint = new Endpoint({
  path: '/note',
  method: EndpointMethod.POST,
  handler: (req, res, next) => res.json(addNote(req.body)),
  authType: EndpointAuthType.AUTH_JWT,
  middleware: [checkPermission(['DeleteUser', 'AdminAccess'])],
  errorMiddleware: [addNoteErrorMiddleware],
  validator: {
    id: {
      in: 'body',
      isUUID: true
    },
    content: {
      in: 'body',
      isLength: {
        errorMessage: 'content to long',
        options: {
          max: 150
        }
      }
    }
  }
});

Methods

The following HTTP methods are supported

• Method.GET
• Method.POST
• Method.PUT
• Method.PATCH
• Method.DELETE
• Method.ALL - respond to all requests on a path

Authentication

Endpoints can take an authType and authParam in their configuration to determine their authentication behavior. The following table summarizes their usage.

The server engine exposes an enumeration for auth types. import {AuthType} from 'node-server-engine';

File Upload Middleware

This middleware handles multipart file uploads in an Express application. It processes files in memory, validates them based on configuration options, and ensures that required files are uploaded.

Usage

The request must be made using multipart/form-data. The uploaded files will be available in req.files.

The following settings can be used on each object the Endpoint's options.files.

Example

import { body } from 'express-validator';
import { Endpoint, middleware, AuthType, Method } from 'node-server-engine';

const filesConfig = [
  { key: 'avatar', mimeTypes: ['image/png', 'image/jpeg'], required: true },
  { key: 'document', mimeTypes: ['application/pdf'], maxSize: '5MB' }
];

new Endpoint({
  path: '/upload',
  method: Method.POST,
  authType: AuthType.JWT,
  files: filesConfig,
  handler: (req, res) => {
    res.json({ message: 'Files uploaded successfully', files: req.files });
  }
});

Features

• Supports multiple file uploads.
• Validates file types and sizes.
• Ensures required files are uploaded.
• Uses memory storage (files are not saved to disk).

This middleware simplifies file handling in Express, making it easy to manage uploads while enforcing validation rules.

Socket Client

The socket server start if the Server's webSocket option is not undefined.

Each time a new socket connection is established, a new instance of SocketClient is created.

Socket Client Options

Options can be set for SocketClient when setting up a Server instance.

Properties

Message Handler

Message handler are similar to Endpoint, but in a WebSocket context. They define how incoming messages should be handled.

import { MessageHandler, Server } from 'node-server-engine';

function handler(payload, client) {
  // payload is the message payload in a standard message.
  // client is an instance of SocketClient
}

const messageHandler = new MessageHandler(type, handler, options);

new Server({
  webSocket: { client: { handlers: [messageHandler] } }
});

Redis

The server engine exposes a Redis client that is configured to work with the standard environment variables that are used in our services.

It is a pre-configured instance of ioredis. See the package documentation for more details.

import { Redis } from 'node-server-engine';

await Redis.set(key, value);

It can be configured through environment variables

Sequelize

The server engine exposes an SQL ORM that is configured to work with the standard environment variables that are used in our services.

It is a pre-configured instance of sequelize. See the package documentation for more details.

import { Sequelize } from 'node-server-engine';

Sequelize.sequelize;
Sequelize.closeSequelize();

It can be configured through environment variables

Pub/Sub

The engine exposes a PubSub entity that can be used to communicate with Google Cloud Pub/Sub.

import { PubSub } from 'node-server-engine';

/**
 * Declare that the service will be publishing to a topic
 * This must be done before init() is called
 * Any attempt to publish a message without declaring a topic first will fail
 * @property {string|Array.<string>} topic - The topic(s) to which we will be publishing
 */
PubSub.addPublisher(topic);

/**
 * Binds a message handle to a subscription
 * If called multiple times, handlers will be chained
 * This must be done before init() is called
 * The subscription will not be consumed until init() is called
 * @property {string} subscription - The subscription to consume
 * @property {function|Array.<function>} handler - The message handling function(s)
 * @property {boolean} [first] - Puts the handler(s) at the beginning of the handling chain (default: false)
 */
PubSub.addSubscriber(subscription, handler, first);

/**
 * Establish connection with all the declared publishers/subscribers
 */
await PubSub.init();

/**
 * Send a message through a previously declared publisher
 * @property {string} topic - The name of the topic to which the message should be pushed
 * @property {Object} message - The actual message (will be JSON stringified)
 * @property {Object} [attributes] - Message attributes
 * @property {string} [orderingKey] - Ordering key
 */
await PubSub.publish(topic, message, attributes, orderingKey);

/**
 * Flush all pending messages and close connections with Pub/Sub
 */
await PubSub.shutdown();

PushNotification

Communication interface with the push service.

import { PushNotification } from 'node-server-engine';

// The entity needs to initialize it's pub/sub connections
// Handlers must be declared before this function is called
await PushNotification.init();

/**
 * Send a push notification through the push service
 * @param {String} userId - ID of the user that should receive the notification
 * @param {Object} notification - Notification that should be sent
 * @return {void}
 */
await PushNotification.sendPush(userId, notification);

Localizator

The localizator exposes localization related utilities.

import { Localizator } from 'node-server-engine';

// The synchronize init should be called first to initialize the data
// The localizator will regularly synchronize new data after that without any calls having to be made
await Localizator.init();

// Get the different ways of displaying a user's name
const { regular, formal, casual } = await Localizator.getDisplayNames(
  firstName,
  lastName,
  locale
);

// This should be call when the program shuts down.
await Localizator.shutdown();

ElasticSearch

The ElasticSearch exposes a service related to data searching.

import { ElasticSearch } from 'node-server-engine';

// The synchronize init should be called first to initialize the data
await ElasticSearch.init();

// This should be call when the program shuts down.
await ElasticSearch.shutdown();

Translation Manager

The translation manager exposes translation related utilities.

import { TranslationManager } from 'node-server-engine';

// The synchronize init should be called first to initialize the data
// The tranlsation manager will regularly synchronize new data after that without any calls having to be made
await TranslationManager.init();

// Get the different ways of displaying a user's name
const translatedString = await TranslationManager.translate(
  lang,
  key,
  variables,
  tags
);

// Example
const translatedString = await TranslationManager.translate(
  'zh-TW',
  'email.invitation.body',
  { name: 'John' },
  { link: ['a', 'href="https://www.test.com"'] }
);

// This should be call when the program shuts down.
await TranslationManager.shutdown();

• lang String: Locale for which the translation should be fetched (if no data is found, translation will be returned in en-US).
• key String: Translation key
• variables Object: A key=>value mapping for variable interpolation in strings. (Optional)
• tags Object: A key=>value mapping for variable interpolation in strings. (Optional)

Error Reporting

The server engine standardizes the way errors are handled and reported. The error classes provided by the Server Engine should always be used when throwing an exception.

Errors are a crucial part of the application, they are what helps us to properly debug the program and offer support when need, as well as what exposes issues to the client.

By standard, the client receives the following body when an error happens.

// HTTP Status code (400 | 500)
{
  errorCode: "some-error", // Machine readable error code
  hint: "The selected user does not exist" // (optional) Hint for developers
}

Status codes should be limited to 400 for client errors and 500 for server errors. Other 4XX status codes should be avoided unless very specific cases (ex: authentication error).

All our custom error classes take a data parameter. This will be logged on the backend and should contain any data that can help to understand the runtime context or the error (ex: user's ID).

Common options

These options are common to each of the error classes described below.

Severity

Severity allows us to order errors by their impact on the program. It is important to set severity correctly as backend logs can include hundreds of entries per seconds, severity allows us to filter out the most important errors. An enumeration is exposed that includes all the severity levels, as described in the following table.

Log Levels

EngineError

This error class represents errors that happen withing the Server Engine. They should be used for server configuration errors, or unexpected behaviors. They will always return 500 - {errorCode: 'server-error'}.

import { EngineError, Severity } from 'node-server-engine';

if (!process.env.IMPORTANT_VARIABLE) {
  throw new EngineError(options);
}

Engine Errors strictly follow the common options.

WebError

This error class represents errors that happen at runtime and that need specific reporting to the client. Their definition is more complex, but it includes additional data specific to the client.

import { WebError, Severity } from 'node-server-engine';

const user = User.findOne({ where: { id: userId } });
if (!user) {
  throw new WebError(options);
}

In addition to the common options, WebError defines some other options specific for error reporting to clients.

Middlewares

Server engine exposes a bunch of middlewares. These can be imported to your project and used globally or per endpoint.

Swagger Docs

This middleware allows the service to connect with the API Documentation Service. It exposes the necessary endpoint for the documentation of this API to be visible by the Documentation Service.

import { Server, middleware } from 'node-server-engine';

new Server({
  globalMiddleware: [middleware.swaggerDocs()]
});

Structuring Documentation

The underlying system used is the Open API spec. Most of the data is already generate by the documentation service. The only real need is to document endpoints.

Endpoints should be documented in YAML files that respect the **/*.docs.yaml convention. It is recommended to place them in the same directory as the endpoint definition. Endpoint documentation has to follow the path object spec from Open API.

/hello:
  get:
    description: Request the API to say Hello
    responses:
      '200':
        description: Replies hello
        content:
          application/json:
            schema:
              type: Object
              properties:
                says:
                  type: string
                  example: Hello

Schemas and Responses

To avoid repeating the same structure across the documentation manually, one can use schemas and responses components.

Some common components are already defined directly in the API Documentation Service, please check its documentation to avoid repeats.

If you ever need to declare custom components, they simply must follow the architecture bellow.

# Repository root
/src
  /docs
    /responses
      - coolResponse.yaml
    / schemas
      - bestSchema.yaml

Here is an example definition:

Dog:
  type: object
  properties:
    name:
      type: string
      example: Rex
    owner:
      type: string
      example: f1982af0-1579-4c56-a138-de1ab4ff39b3
    isAGoodBoy:
      type: boolean
      example: true
  required:
    - name
    - owner

User Resolver

/!\ Must be used in combination with AuthType.JWT

Resolves the user making the request with the user resolver. The user's complete data is added to req.user.

import { Endpoint, middleware, AuthType, Method } from 'node-server-engine';

new Endpoint({
  path: '/hello',
  method: Method.GET,
  authType: AuthType.JWT,
  middleware: [middleware.userResolver],
  handler: (req, res) => {
    res.json({ say: `Hello ${req.user.firstName}` });
  }
});

Gemini File Upload

An endpoint can upload a file to a google gemini AI

The request must be made as multipart/form-data.

File should be uploaded under the key file.

The file's data will be available at req.body.fileUri req.body.mimeType req.body.originalname

Multipart File Uploader

This middleware handles multipart file uploads by accepting file chunks, storing them temporarily, and merging them once all chunks are uploaded.

Usage

The request must be made using multipart/form-data. The file should be uploaded under the key file.

Once the upload is complete, the following data will be available in req.body.uploadDetails:

• status: "pending" (if chunk upload is in progress) or "completed" (if file is fully uploaded and merged).
• message: Status message.
• details:
  • filename: Original filename.
  • uniqueID: Unique identifier for the upload session.
  • chunkIndex: The current chunk index (if upload is still in progress).
  • totalChunks: Total number of chunks for the file (if upload is still in progress).

Request Format

Endpoint:

POST /upload

Headers:

Content-Type: multipart/form-data

Form Data:

Response Format

If chunk upload is successful but not completed:

{
  "status": "pending",
  "message": "Chunk uploaded successfully",
  "details": {
    "filename": "example.pdf",
    "uniqueID": "123456",
    "chunkIndex": 2,
    "totalChunks": 5
  }
}

If all chunks are uploaded and merged successfully:

{
  "status": "completed",
  "message": "File uploaded and merged successfully",
  "details": {
    "filename": "example.pdf",
    "uniqueID": "123456"
  }
}

Error Handling

If an error occurs (e.g., missing fields, failed write operations), the middleware will return an error response with details.

Example:

{
  "status": "failed",
  "message": "Missing required fields",
  "details": {
    "filename": null,
    "totalChunks": null,
    "chunkIndex": null,
    "uniqueID": null,
    "receivedChunk": false
  }
}

This middleware ensures smooth handling of large file uploads by splitting them into smaller chunks and merging them once all chunks are received. 🚀

Check Permission Middleware

/!\ Must be used in combination with AuthType.JWT

Checks if the user has at least one of the required permissions. The permissions are case-insensitive.

import { Endpoint, middleware, AuthType, Method } from 'node-server-engine';

new Endpoint({
  path: '/admin',
  method: Method.GET,
  authType: AuthType.JWT,
  middleware: [middleware.checkPermission(['admin', 'superuser'])],
  handler: (req, res) => {
    res.json({ message: 'Access granted' });
  }
});

import { Request, Response, NextFunction } from 'express';

// Middleware to check if the user has at least one of the required permissions (case-insensitive)
export const checkPermission = (requiredActions: string | string[]) => {
  return (req: Request, res: Response, next: NextFunction) => {
    const user = req.user;

    if (!user || !user.permissions) {
      return res
        .status(403)
        .json({ message: 'User does not have permissions' });
    }

    const requiredActionsArray = Array.isArray(requiredActions)
      ? requiredActions
      : [requiredActions];

    const requiredActionsLower = requiredActionsArray.map((action) =>
      action.toLowerCase()
    );

    const hasPermission = user.permissions.some((permission: string) =>
      requiredActionsLower.includes(permission.toLowerCase())
    );

    if (!hasPermission) {
      return res.status(403).json({ message: 'Permission denied' });
    }

    next();
  };
};

Utilities

The server engine ships with a handful of utility functions that are commonly used by servers

Request

This function is a wrapper around axios. It adds proper error handling for reporting in the logs when a network call fails. It should be used for any requests made by a service. Refer to the axios documentation for the request configuration.

import { request } from 'node-server-engine';
const { data } = await request({
  method: 'get',
  url: 'https://www.google.com'
});

TLS Request

This function is a wrapper around request. It adds the necessary configuration to easily make requests with TLS. The settings used are based on the environment variables. It will use request specific certificate/key if defined, or will fallback to use the same ones as the ones used for the server.

It is necessary to use this function when calling other services in the cluster. Requests could fail otherwise as the common CA is not set and the client certificate not exposed.

import { tlsRequest } from 'node-server-engine';
const { data } = await tlsRequest({
  method: 'get',
  url: 'https://www.google.com'
});

TLS Config

The server's TLS configuration can be fetched as an object. Alternatively, the server engine also exposes an HTTPS Agent. They will not be available at startup, so it is important that the function calling them loads them first if they are not present.

import {tlsConfig, httpsAgent, loadTlsConfig} from 'node-server-engine';
import https from 'https';

// TLS Config
if(!tlsConfig) loadTlsConfig();
const customAgent = new https.Agent({
  key: tlsConfig.key,
  cert: tlsConfig.cert,
  ca: tlsConfig.ca,
  passphrase: tlsConfig.passphrase,
});

// HTTPS Agent
if(!httpsAgent) loadTlsConfig();
https.request('https://www.google.com'. {agent: httpsAgent});

Send Push Notification

Send a push notification through the push service.

import { sendPush } from 'node-server-engine';
await sendPush(userId, notification);

Send Email

Send an email using the SMTP configuration.

import { sendEmail } from 'node-server-engine';

const emailOptions = {
  to: 'recipient@example.com',
  subject: 'Welcome to our service!',
  text: 'Hello, welcome to our platform!',
  html: '<h1>Welcome</h1><p>Glad to have you onboard!</p>',
  attachments: [
    {
      filename: 'welcome.txt',
      content: 'Welcome to our service!'
    }
  ]
};

const result = await sendEmail(emailOptions);
console.log(result.status); // 'sent', 'delivered', 'queued', or 'failed'

Parameters

Return Status

The function returns an object with the following status options:

Gemini File Upload

Upload a file to Google Gemini AI.

import { geminiFileUpload } from 'node-server-engine';

const fileBuffer = fs.readFileSync('example.pdf');
const mimeType = 'application/pdf';
const originalName = 'example.pdf';

const result = await geminiFileUpload(fileBuffer, mimeType, originalName);

if (result.success) {
  console.log('File uploaded successfully:', result.fileUri);
} else {
  console.error('File upload failed:', result.error);
}

Parameters

Response

The function returns an object with one of the following structures:

Success Response

{
  "success": true,
  "originalname": "example.pdf",
  "fileUri": "https://gemini.googleapis.com/file/xyz123",
  "mimeType": "application/pdf"
}

Failure Response

{
  "success": false,
  "error": "Error message"
}

Return Fields

Error Handling

• If the GOOGLE_AI_KEY environment variable is missing, the function throws an error.
• If the upload fails or the file processing does not complete successfully, an error response is returned.
• Temporary files are cleaned up after the upload process to prevent storage issues.

Filter

Apply a filter on an object. It returns a copy of the object that only holds the whitelisted keys.

This is particularly useful to sanitize objects before returning them to clients.

import { filter } from 'node-server-engine';

const object = { a: 'kept', b: 'not kept' };
const whitelist = ['a'];

const result = filter(object.whitelist);
// result = {a: 'kept'}

Database Migration

Some utilities are exposed to handle database migrations.

runPendingMigrations will execute all the migration scripts that have not yet been executed.

rollbackMigrations will rollback all the migrations that have been executed with the current version of the app and after.

import { runPendingMigrations } from 'node-server-engine';
import { rollbackMigrations } from 'node-server-engine';

await runPendingMigrations();

await rollbackMigrations();

Environment Variables Verification

Environment variables verification can be done through the Server's checkEnvironment setting. It is an object defining how environment variables should be verified.

import { envAssert } from 'node-server-engine';

export const checkEnvironment = {
  ENV_VAR: envAssert.isString()
};

Assertions Available

The sever engine makes available a utility for environment variables assertions calls envAssert. The following example shows the different assertions that are possible.