qpp-shared-logger-node v2.2.3

qpp-shared-logger-node

A simple configurable wrapper around the winston application logger and the morgan http access logger.

This logger is opinionated. Configure it with a minimal set of options, and get Splunk-searchable, PII-scrubbed, and QPP-compliant logs with minimal effort.

Installation

$ npm install qpp-shared-logger-node --save

Usage

This module accepts configuration, sets up the logger, then yields a winston Logger object for you to use.

It is recommended that you set up your project with a log.js module that you can import from the rest of your project; log.js is where you will configure the shared logger and expose the winston Logger.

First, pass your configuration to the module, then get access to the Logger object.

log.js:

const sharedLogger = require('qpp-shared-logger-node');

sharedLogger.configure({
    projectSlug: 'my-app-name'
});

module.exports = sharedLogger.logger; // a winston Logger

In other parts of your code:

const logger = require('log');

logger.info('fetching data', { id: object.id, sort: 'asc' });

Logging with context fields

The contextLogger function returns a logger object that includes a set of fields with each log message. The fields are merged with those passed to a logging method.

For example, this can be used to log a request id in all entries related to a single client request:

req.logger = sharedLogger.contextLogger({ requestId: uuidv1(), url: req.url });
req.logger.info('started');
req.logger.error(message, { error: err });

Field values in the log call take priority over values in the context fields.

Context Logging Defaults

There are defaults that should be added to the context of the beginning with each request if possible.

const requestContext = {
  requestId: uuidv1(),
  oktaId: AuthService(), // The auth service should return the okta id.
  applicationBuild: process.env.BUILD_ID // what constant the application would be storing this information
}

req.logger = sharedLogger.contextLogger(requestContext);

Configuration

Environment Defaults

This module will read process.env.NODE_ENV to determine your application's "environment", and configure these settings out-of-the-box:

Advanced Application Log Configuration

You may override the defaults:

Splunk Transport Configuration

WARNING - If the Splunk transport fails to connect to Splunk, log messages will be lost during the outage. DO NOT rely on this logger alone if you need to have guaranteed delivery of all logs to Splunk.

Advanced HTTP Access Log Configuration

Override defaults.

Considerations for production logging

• Splunk ingesters are set up to find log files under /var/log, so always send your production logs here or to a subdirectory within.

• To differentiate logs emitted from the DEV or IMP or PROD deployed environments, be sure the name is embedded in one or more of the Splunk-searchable fields. For example, configure your log directory with the "deployed environment" embedded as a subdir: /var/log/qpp-service/{{ dev|imp|prod }}/{{ projectSlug }}/

  So on Halloween, Splunk will index the source of log messages as /var/log/qpp-service/imp/projectSlug/app.20171031.log. A sample Splunk query to pick up your IMP logs: source=/var/log/qpp-service/imp/projectSlug/*.

• Suggestion: use an environment var (not NODE_ENV) to communicate the name of the environment from the OS to the app. The auth service, for example, uses the ENVIRONMENT var.

PII (Personally-Identifiable Information) and log scrubbing

Log messages are composed of a message (String) and metadata (key-value). Keys in the metadata that match one of the redactKeys will not have their value logged. Note that the logger does not search the String message for data to scrub. Please keep log messages simple, and add metadata with proper keys.

Compatibility

Tested with node v10.15.3 and node v12.15.0

Development

$ brew install gitleaks # Installs gitleaks for pre-commit secret detection. If not running on Mac, see https://github.com/zricethezav/gitleaks/releases
$ npm install       # install dependencies
$ npm  test         # run tests, also report coverage in ./coverage/index.html
$ npm run  format   # run prettier to format code
$ npm run lint      # run eslint to check code

Release Process

The release process is semi-automated via github actions. A number of steps are necessarily left manual (such as versioning) and require intervention from the user.

1. Create a release branch release/* either off of master to pull all changes, or by cherry-picking only certain changes.

2. Bump the version using npm version <patch | minor | major>.

3. Push the release branch to github.

4. Github actions will automatically create a release and tag based off the version in package.json.

5. Review the draft release page and publish it as a pre-release.

6. Github actions will automatically publish a package to npm. Additionally, a new pull request will be created to backfill master from release if necessary.

Want to Contribute?

Want to file a bug or contribute some code? Read up on our guidelines for contributing.

Public Domain

This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication.

All contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.

See the formal LICENSE file.

Resources

• https://jira.cms.gov/browse/QPPELIG-130
• https://jira.cms.gov/browse/QPPELIG-131
• https://confluence.cms.gov/display/QPPGUIDE/Shared+Logger
• https://confluence.cms.gov/display/QPPPLAT/Logging+Standard+Proposal
• https://confluence.cms.gov/display/QPPGUIDE/Shared+Library+V1+Milestone
• https://www.owasp.org/index.php/Logging_Cheat_Sheet
• https://www.npmjs.com/package/winston
• https://www.npmjs.com/package/morgan
• https://github.com/i0natan/nodebestpractices/blob/master/sections/production/setnodeenv.md