@achs/env v4.8.0

📖 About

Eases NodeJS environment variable handling, like env-cmd or dotenv, but with powerfull features and extensibility for adding custom providers (as plugins) for load, pull and push the variables from different stores.

📌 Requirements

First, download and install NodeJS. Version 16 or higher is required.

Validate installed versions of node and npm with:

> node -v
v16.14.2

> npm -v
8.3.0

You can initialize a new npm project using:

> npm init

⚡️ Quick start

🔔 Make sure that you have NodeJS 14+ installed on your computer.

• Installs the package:

> npm install @achs/env

  added 1 packages, and audited 1 packages in 1s

  found 0 vulnerabilities
> _

• Executes binary directly:

> node_modules/.bin/env --help

  Usage: env [command] [options..] [: subcmd [:]] [options..]

  Commands:
	env [options..] [: <subcmd> :]
	env pull [options..]
	env push [options..]
	env schema [options..]
> _

> npx env --help

  Usage: env [command] [options..] [: subcmd [:]] [options..]

  Commands:
	env [options..] [: <subcmd> :]
	env pull [options..]
	env push [options..]
	env schema [options..]
> _

• Or add desired commands in your npm script in package.json:

{
  ...,
  "scripts": {
	// starts project injecting "dev" environment variables and debug log level
	"start:dev": "env -e dev -m debug : node dist/main.js : --log debug",
	// starts project injecting "prod" environment variables
	"start:prod": "env -e prod -m debug : node dist/main.js",
	...,
	// builds project injecting "prod" environment variables
	"build:prod": "env -e prod -m build : tsc",
	...,
	"env:schema": "env schema -e dev --ci",
	// uploads environment "dev" variables
	"env:push:dev": "env push -e dev",
	// downloads environment "dev" variables
	"env:pull:dev": "env pull -e dev"
  },
  ...
}

• Execs your command:

file: dist/main.js

console.log(`My environment loaded is: ${process.env.ENV}`);

> npm run start:dev

  13:31:59.865 INFO  loading dev environment in debug mode
  13:31:59.911 DEBUG using package-json provider
  13:31:59.912 DEBUG using app-settings provider
  13:31:59.914 DEBUG using secrets provider
  13:32:00.109 DEBUG environment loaded:
  {
	NODE_ENV: 'development',
	ENV: 'dev',
	VERSION: '1.0.0',
	NAME: '@my-app',
	VAR1: true,
	VAR2: true,
	GROUP1__VAR1: 'G1V2',
	ARR1: '1,val,true',
	SECRET: '***'
  }
  13:32:00.116 INFO  executing command > node dist/main.js
  My environment loaded is: dev
  13:32:00.232 INFO  process finished successfully
> _

⛩ Structure

├── src/
│   ├── commands/ # lib commands handlers
│   │   ├── env.command.ts
│   │   ├── export.command.ts
│   │   ├── pull.command.ts
│   │   ├── push.command.ts
│   │   └── schema.command.ts
│   ├── interfaces/ # provider interfaces
│   ├── providers/ # integrated providers
│   │   ├── package-json.provider.ts
│   │   ├── app-settings.provider.ts
│   │   ├── local.provider.ts
│   │   └── azure-key-vault.provider.ts
│   ├── utils/
│   │   ├── command.util.ts
│   │   ├── interpolate.util.ts
│   │   ├── json.util.ts
│   │   ├── normalize.util.ts
│   │   ├── schema.util.ts
│   │   └── logger.ts
│   ├── arguments.ts # global arguments
│   ├── exec.ts # initialization logic (load config, commands, etc.)
│   └── main.ts
├── tests/ # integration tests
├── .eslintrc.json
├── jest.config.json
├── tsconfig.build.json
└── tsconfig.json

⚙️ Commands & Options

Options handling has the ability of replace arguments itself, using [[ and ]] as delimiters. So, in example for define your config file path, you must use your root argument, supposing root has the value of "config", this definition [[root]]/any-config-file.json will be config/any-config-file.json, or if your env argument is "dev", this definition [[root]]/config-file.[[env]].json will be config/config-file.dev.json.

Global options

Workspace options

JSON Schema options

Logger options

• env

Inject your environment variables into process.env and executes a command.

env -e [env] [options..] [: subcmd [:]] [options..]

Examples:

> env -e dev -m test unit : npm test

> env -e dev -m debug : npm start : -c [[root]]/[[env]].env.json

> env -e prod -m build optimize : npm build

• pull

Pulls environment variables from providers stores.

env pull -e [env] [options..]

Examples:

> env pull -e dev

• push

Pushes environment variables to providers stores.

env push -e [env] [options..]

Examples:

> env push -e dev

• schema

Generates validation schema from providers output variables.

env schema -e [env] -m [modes] [options..]

Examples:

> env schema -e dev -m build

• export

Export unified environment variables to a file from providers.

env export -e [env] -m [modes] [options..]

Examples:

> env export -e dev -m build -f json --uri [[env]].env.json

📡 Providers

Main feature of this library is using providers for get and set environment variables. So, you can define your own provider, but lib came with 3 integrated providers:

• package-json

Load some info from your project package.json.

Info read is:

{
	"version": "1.0.0",
	"project": "project-name",
	"name": "@package-name",
	"title": "app-name",
	"description": "any description"
}

Examples:

> env -e dev -m build : react-script build : --vp REACT_APP_

• app-settings

Non secrets loader for appsettings.json.

appsettings.json file has the format below:

{
	"|DEFAULT|": {},
	"|MODE|": {},
	"|ENV|": {}
}

In example:

{
	"|DEFAULT|": {
		"VAR1": "v1_default"
	},
	"|MODE|": {
		"build": {
			"NODE_ENV": "production"
		},
		"debug": {
			"NODE_ENV": "development"
		},
		"test": {
			"NODE_ENV": "test"
		}
	},
	"|ENV|": {
		"dev": {
			"C1": "V1",
			"C2": "V2",
			"C3": 3,
			"GROUP1": {
				"VAR1": null,
				"VAR2": "G1V2",
				"VAR3": true,
				"GROUP2": {
					"VAR1": "G1G2V1"
				}
			},
			"C4": "23"
		}
	}
}

• package-json

Load some info from your project package.json.

Info read is:

{
	"version": "1.0.0",
	"project": "project-name",
	"name": "@package-name",
	"title": "app-name",
	"description": "any description"
}

Examples:

> env -e dev -m build : react-script build : --vp REACT_APP_

• azure-key-vault

Azure Key Vault provider, allows to load secrets from vault store to env/secrets/[[env]].env.json per environment. Also, handles env/secrets/[[env]].local.env.json for load local variables with precedence over base.

✒ Creating Custom Providers

You can create your custom providers, in two ways:

• Local Script: you must create a JavaScript file (.js), exporting by default your "provider" following standard interface exported by this lib.
• NPM Package: you must create your custom NPM library and export by default your "provider" using standard interface exported by this lib.

How to load your provider is shown in Config Section.

In example, a provider exported by your NPM package written in TypeScript should be like:

import { CommandArguments, EnvProvider } from '@achs/env';
import { logger, readJson, writeJson } from '@achs/env/utils';

const KEY = 'my-unique-provider-key';

interface MyProviderCommandArguments extends CommandArguments {
	anyExtraOption: boolean;
}

export const MyProvider: EnvProvider<MyProviderCommandArguments> = {
	// unique identifier for provider
	key: KEY,

	// (optional) allows to provider adds new arguments/options
	// to commands using yargs for builder
	builder: (builder) => {
		builder.options({
			anyExtraOption: {
				group: KEY,
				alias: ['a', 'aeo'],
				type: 'boolean',
				default: false,
				describe: 'Any option description',
			},
		});
	},

	// call on environment variables loading,
	// may be a Promise
	load: ({ env, modes, ...options }) => {
		if (env === 'dev')
			return {
				NODE_ENV: 'development',
			};

		// you can return a list of JSON environment variables for merge
		return [
			{
				NODE_ENV: 'production',
			},
			{
				ANY_VAR: 'ANY_VALUE',
				ANY_GROUP: {
					INNER_VAR: 12,
				},
			},
		];
	},

	// (optional) call on pulling variables from provider store,
	// config may pass in your config file
	pull: ({ env, modes, ...options }, config) => {
		// anyway you want for pulling variables to cache
	},

	// (optional) call on pushing/updating variables to provider store,
	// config may pass in your config file
	push: ({ env, modes, ...options }, config) => {
		// anyway you should do for pushing or updating your variables
	},
};

📥 Config

You can configure any config argument inside you config file, but commonly providers are designed for this purpose.

{
	"log": "silly",
	// will hide values of keys SECRET and MY_API_KEY in logging
	"logMaskValuesOfKeys": ["SECRET", "MY_API_KEY"],
	// integrated providers and custom providers together
	"providers": [
		{
			"path": "package-json"
		},
		{
			"path": "app-settings"
		},
		{
			"path": "azure-key-vault",
			"config": {
				"dev": {
					"vaultUrl": "https://kv-desa-ittec-sti.vault.azure.net"
				},
				"qa": {
					"vaultUrl": "https://kv-qa-ittec-sti.vault.azure.net"
				}
			}
		},
		{
			"path": "local"
		},
		{
			// custom NPM package
			"path": "@npm-package",
			"type": "module",
			"config": {
				"any-config": "any value"
			}
		},
		{
			// custom script inside project
			"path": "scripts/custom-loader.js",
			"type": "script"
		}
	]
}

💽 Azure Key Vault

Allows you to store your secrets in Azure Key Vault.

1.2. NPM Scripts

For load desired environment, add you npm script like env -e <env> -m <mode1[ mode2]> : <your-command>.

• mode: (build|debug|test) execution mode base variables.
• env: (dev|qa|stg|prod) environment variables.

In example: env -e dev -m debug : npm start

2. Structure

2.1. Environments

Your env/secrets folder will contain files below:

• dev.env.json: development environment.
• dev.local.env.json: local development environment (takes precedence).
• qa.env.json: quality assurance environment.
• qa.local.env.json: local qa environment (takes precedence).
• prod.env.json: production environment.
• prod.local.env.json: local production environment (takes precedence).

This folder should contains environment variables files for system environments.

2.2. Keys (env/keys.json)

Your keys.json file should contains you Azure Key Vault SPN credentials per environment:

{
	"<env-name>": {
		"vaultUrl": "<azure-key-vault-url>", // you can skip this var if present in config
		"clientId": "<spn-client-id>",
		"clientSecret": "<spn-secret-password>",
		"tenantId": "<tenant-id>"
	},
	...
}

In example:

{
	"dev": {
		"clientId": "f176a774-239e-4cd3-8551-88fd9fb9b441",
		"clientSecret": "WyBwkmcL8rGQe9B2fvRLDrqDuannE4Ku",
		"tenantId": "6d4bbe0a-5654-4c69-a682-bf7dcdaed8e7"
	},
	"qa": {
		"clientId": "5dcd9f45-7067-4387-94d8-e5e7066ba630",
		"clientSecret": "60ec5e16430a46eba70dfea80d721b66",
		"tenantId": "6d4bbe0a-5654-4c69-a682-bf7dcdaed8e7"
	}
}

Your secrets will be grouped, using your "name" and "project" variables from package.json file.

This file allows to load environment files locally first run time.

3. Commands

You can use two command scripts for refresh your local env files or publish/updates env files in the azure key vault from your local files.

• Pulls Secrets File: env pull -e <env> [-o]. (-o forces to replace your local file).
• Pushes/Publishes Secrets File: env push -e <env>.

3.1. Environment Variables for Credentials

You can set your credentials variables from node environment variables.

In example:

user@machine:/mnt/c/Users/user$ AZURE_VAULT_URL=https://kv-desa-ittec-sti.vault.azure.net \
								AZURE_CLIENT_ID=f176a774-239e-4cd3-8551-88fd9fb9b441 \
								AZURE_CLIENT_SECRET=WyBwkmcL8rGQe9B2fvRLDrqDuannE4Ku \
								AZURE_TENANT_ID=6d4bbe0a-5654-4c69-a682-bf7dcdaed8e7 \
								node_modules/.bin/env pull -e dev -o

or

user@machine:/mnt/c/Users/user$ npx pull -e dev -o \
								--vaultUrl https://kv-desa-ittec-sti.vault.azure.net \
								--spn f176a774-239e-4cd3-8551-88fd9fb9b441 \
								--password WyBwkmcL8rGQe9B2fvRLDrqDuannE4Ku \
								--tenant 6d4bbe0a-5654-4c69-a682-bf7dcdaed8e7

4. Schema

This lib uses JSON schema for validate and retrieve secrets from store.

For each property in the file, loader will retrieve the value from Azure Key Vault.

When you push a new variable from any of your environment secrets file, env.schema.json will be updated automatically.

If you want to ignore to load some variable without delete it, you can remove the variable from env.schema.json.

5. Shared and Nested Keys

You can organize your keys in nested objects, and declare project shared secrets (skips group separation) prefixing with '\$' like:

{
	// .dev.env.json
	"$SHARED": "sharedValue",
	"GROUP1": {
		"$SHARED": "sharedValue2",
		"VAR": "anyValue1",
		...
	},
	"GROUP2": {
		"VAR": "anyValue2",
		"SUBGROUP1": {
			"VAR": "anyValue1",
			...
		},
		...
	},
	"VAR3": "anyValue3",
	...
}

So, in your application you can use the variables as shown below:

const myVar1 = process.env.GROUP1__VAR;
const myVar2 = process.env.GROUP2__VAR;
const myVar2 = process.env.GROUP2__SUBGROUP1_VAR;
const myVar3 = process.env.VAR3;
// shared vars will load in every project
const mySharedVar1 = process.env.SHARED;
const mySharedNestedVar2 = process.env.GROUP1__SHARED;

6. Priority

From lowest to highest.

• (dev|qa|prod).env.json
• (dev|qa|prod).local.env.json (takes precedence over previous)

🧿 Linting

Project uses ESLint, for code formatting and code styling normalizing.

• eslint: linter integrated with TypeScript.

For correct interpretation of linters, is recommended to use Visual Studio Code as IDE and install the plugins in .vscode folder at 'extensions.json', as well as use the config provided in 'settings.json'

📋 Changelog

For last changes see CHANGELOG.md file for details.

🛠️ Built with

• yargs
• tslog
• subslate
• merge-deep
• ajv
• to-json-schema