incountry v5.1.0

InCountry Storage SDK

Table of contents

• Installation
• Countries list
• Quickstart guide
• Storage configuration
  • oAuth options configuration
  • Encryption key/secret
  • Server side encryption
  • Logging
  • Skipping Storage validation
• Usage
  • Writing data to storage
    • List of available record fields
    • Date fields
  • Batches
  • Reading stored data
  • Find records
    • Exact match search
    • Partial text match search
    • Search options
      • Sorting
    • Find error handling
  • Find one record
  • Deleting a single record
  • Deleting multiple records
• Attaching files to a record
  • Adding attachments
  • Deleting attachments
  • Downloading attachments
  • Working with attachment meta info
• Health check
• Data migration and key rotation support
• AWS KMS integration
• Error handling
• Custom encryption support
• Testing Locally

Installation

The SDK is available via NPM:

npm install incountry --save

Countries List

To get the full list of supported countries and their codes, please follow this link.

Quickstart guide

To access your data in InCountry Platform by using NodeJS SDK, you need to create an instance of the Storage class using the createStorage async factory method. You can retrieve the oauth.clientId, oauth.clientSecret and environmentId variables from your dashboard on InCountry Portal.

const { createStorage } = require('incountry');
const storage = await createStorage({
  environmentId: '<environment_id>',
  oauth: {
    clientId: '<client_id>',
    clientSecret: '<client_secret>',
  },
  getSecrets: () => '<encryption_secret>',
});

Storage Configuration

Below you can find a full list of possible configuration options for creating a Storage instance.

type StorageOptions = {
  environmentId?: string;   // Required

  oauth?: {
    clientId?: string;      // Required when using oAuth authorization
    clientSecret?: string;  // Required when using oAuth authorization
    authEndpoints?: {       // Custom endpoints regional map to use for fetching oAuth tokens
      default: string;
      [key: string]: string;
    };
    token?: string;         // Used when OAuth token is already acquired prior to Storage initialization. Mutually exclusive with clientId, clientSecret, authEndpoints
  };

  endpoint?: string;        // Defines API URL
  encrypt?: EncryptionType | boolean;        // Optional. Default is EncryptionType.LOCAL. Switch encryption modes:
  // EncryptionType.LOCAL - Enables encryption using SDK. Encrypted data will be sent to InCountry platform.
  // EncryptionType.SSE - Enables the server side encryption (SSE) mode. Unencrypted data will be sent to InCountry platform via HTTPS, then encrypted and saved on the server side. Enabling this option disables local (SDK-side) encryption, hashing and normalizing.
  // EncryptionType.OFF - Base64 encoded not encrypted data will be sent and stored at InCountry platform
  // Also supports boolean values as backward compatibility:
  // true = EncryptionType.LOCAL
  // false = EncryptionType.OFF

  logger?: Logger;
  getSecrets?: Function;    // Used to fetch an encryption secret. Not compatible with SSE mode.
  normalizeKeys?: boolean;  // If set to 'true', all keys are stored as lowercase. The default value is 'false'. Not compatible with SSE mode.
  countriesCache?: CountriesCache;
  hashSearchKeys?: boolean; // Set to false to enable partial match search among record's text fields `key1, key2, ..., key20`. Defaults to true. Not compatible with SSE mode.

  /**
   * Defines API base hostname part to use.
   * If set, all requests will be sent to https://${country}${endpointMask} host instead of the default
   * one (https://${country}-mt-01.api.incountry.io)
   */
  endpointMask?: string;

  /**
   * If your PoPAPI configuration relies on a custom PoPAPI server (rather than the default one)
   * use the `countriesEndpoint` option to specify the endpoint responsible for fetching the list of supported countries.
   */
  countriesEndpoint?: string;

  httpOptions?: {
    timeout?: NonNegativeInt; // HTTP request timeout, in milliseconds
    maxBodyLength?: NonNegativeInt; // Maximum allowed HTTP request body length, in bytes; Defaults to 100 Mb
    retryBaseDelay?: NonNegativeInt; // Base delay for HTTP request retries in case of server high load, in milliseconds; Defaults to 1s
    retryMaxDelay?: NonNegativeInt; // Maximum delay for HTTP request retries in case of server high load, in milliseconds; Defaults to 32s
  };
};

async function createStorage(
  options: StorageOptions,
  customEncryptionConfigs?: CustomEncryptionConfig[]
): Promise<Storage> {
  /* ... */
}

oAuth options configuration

The SDK allows to precisely configure oAuth authorization endpoints (if needed). Use this option only if your plan configuration requires so.

Below you can find the example of how to create a storage instance with custom oAuth endpoints:

const { Storage } = require('incountry');
const storage = new Storage({
  environmentId: '<environment_id>',
  getSecrets: () => '<encryption_secret>',
  oauth: {
    clientId: '<client_id>',
    clientSecret: '<client_secret>',
    authEndpoints: {
      "default": "<default_auth_endpoint>",
      "emea": "<auth_endpoint_for_emea_region>",
      "apac": "<auth_endpoint_for_apac_region>",
      "amer": "<auth_endpoint_for_amer_region>",
    },
  },
});

The SDK also allows to use previously acquired oAuth tokens if needed. In this mode SDK is not responsible for oAuth token renewal and it should be done by SDK user himself.

Below you can find the example of how to specify OAuth token while creating a Storage instance:

const { Storage } = require('incountry');
const storage = new Storage({
  environmentId: '<environment_id>',
  getSecrets: () => '<encryption_secret>',
  oauth: {
    token: '<token>',
  },
});

Encryption key/secret

The getSecrets storage config option allows you to pass function which will be used to fetch an encryption key(s) or secret. This function should return either a string representing your secret or an object (so-called SecretsData) or a Promise which is resolved to such string or object:

const { createSecretKey } = require('crypto');

type Secret = {
  secret: string | KeyObject;
  version: NonNegativeInt;
  isKey?: boolean;
  isForCustomEncryption?: boolean;
};

type SecretsData = {
  currentVersion: NonNegativeInt;
  secrets: Array<Secret>;
};

/// SecretsData example
{
  secrets: [
    {
      secret: 'aaa',
      version: 0
    },
    {
      secret: 'ABC...IHN0cmluZw==', // Should be a base64-encoded key (32 byte key)
      version: 1,
      isKey: true
    },
    {
      secret: 'ccc',
      version: 2,
      isForCustomEncryption: true
    }
    {
      secret: createSecretKey(Buffer.from('ABC...IHN0cmluZw==', 'base64')), // KeyObject with 32 byte key
      version: 3,
    }
  ],
  currentVersion: 1
};

Note: even though SDK uses PBKDF2 to generate a cryptographically strong encryption key, you must ensure that you provide a secret/password which follows the modern security best practices and standards.

The SecretsData object allows you to specify multiple keys/secrets which the SDK will use for decryption based on the version of the key or secret used for encryption. Meanwhile SDK will encrypt data only by using a key (or secret) which matches the currentVersion parameter provided in the SecretsData object.

This enables the flexibility required to support Key Rotation policies when secrets (or keys) must be changed with time. The SDK will encrypt data by using the current secret (or key) while maintaining the ability to decrypt data records that were encrypted with old secrets (or keys). The SDK also provides a method for data migration which allows you to re-encrypt data with the newest secret (or key). For details please see the migrate method.

The SDK allows you to use custom encryption keys, instead of secrets. Please note that a user-defined encryption key should be a base64-encoded 32-bytes-long key as required by AES-256 cryptographic algorithm.

Below you can find several examples of how you can use the getSecrets storage config option:

// Synchronous
const getSecretsSync = () => 'longAndStrongPassword';
const storage = await createStorage({
  ...,
  getSecrets: getSecretsSync,
});

// Asynchronous
const getSecretsAsync = async () => {
  const secretsData = await getSecretsDataFromSomewhere();
  return secretsData;
};
const storage = await createStorage({
  ...,
  getSecrets: getSecretsAsync,
});

// Using promises syntax
const getSecretsPromise = () =>
  new Promise(resolve => {
    getSecretsDataFromSomewhere(secretsData => {
      resolve(secretsData);
    });
  });
const storage = await createStorage({
  ...,
  getSecrets: getSecretsPromise,
});

Server side encryption

SDK supports server side encryption mode. In this mode all the data encryption and hashing is performed server-side. The input data is sent to the InCountry platform as is. Below you can find an example of how to use the { encrypt: EncryptionType.SSE } option:

const { createStorage } = require('incountry');
const storage = await createStorage({
  environmentId: '<environment_id>',
  oauth: {
    clientId: '<client_id>',
    clientSecret: '<client_secret>',
  },
  encrypt: EncryptionType.SSE,
});

With SSE mode enabled SDK ignores the following configuration parameters: normalizeKeys, hashSearchKeys and getSecrets callback.

NOTE Availability of the SSE mode may vary per subscription plan. Please contact the InCountry support team for details.

Logging

By default, the SDK outputs logs into console in JSON format. You can override this behavior by passing the logger object as a Storage constructor parameter. The logger object must correspond to the following structure:

// Custom logger must implement `write` method

const customLogger = {
  write: (logLevel: LogLevel, message: string, meta?: {}): void => console.log(`${new Date().toISOString()} [${logLevel}] ${message}`, meta);
};

const storage = await createStorage({
  environmentId: 'envId',
  oauth: { clientId: 'clientId', clientSecret: 'clientSecret' },
  getSecrets: () => '',
  logger: customLogger
});

Skipping Storage validation

You can create an instance of the Storage class and run all asynchronous checks by yourself (or skip them at your own risk!).

const { Storage } = require('incountry');
const storage = new Storage({...});

await storage.validate();

The validate method fetches the secret using getSecrets and validates it. If custom encryption configurations were provided they would also be checked with all the matching secrets.

Usage

Writing data to Storage

Use the write method to create/replace a record (by recordKey).

type StorageRecordData = {
  recordKey: string; // Accepts non-empty string
  parentKey?: string | null;
  profileKey?: string | null;
  key1?: string | null;
  ...,
  key20?: string | null;
  serviceKey1?: string | null;
  ...,
  serviceKey5?: string | null;
  body?: string | null;
  precommitBody?: string | null;
  rangeKey1?: number | null; // Accepts only Integer numbers
  ...,
  rangeKey10?: number | null; // Accepts only Integer numbers
  expiresAt?: string | number | Date | MicroDate | null; // Accepts ISO-8601 formatted strings with microseconds, numeric timestamps with decimal microseconds, Date and MicroDate objects
};

type StorageRecord = {
  recordKey: string;
  body: string | null;
  parentKey: string | null;
  profileKey: string | null;
  precommitBody: string | null;
  key1?: string | null;
  key2?: string | null;
  key3?: string | null;
  key4?: string | null;
  key5?: string | null;
  key6?: string | null;
  key7?: string | null;
  key8?: string | null;
  key9?: string | null;
  key10?: string | null;
  key11?: string | null;
  key12?: string | null;
  key13?: string | null;
  key14?: string | null;
  key15?: string | null;
  key16?: string | null;
  key17?: string | null;
  key18?: string | null;
  key19?: string | null;
  key20?: string | null;
  serviceKey1: string | null;
  serviceKey2: string | null;
  serviceKey3: string | null;
  serviceKey4: string | null;
  serviceKey5: string | null;
  rangeKey1: Int | null;
  rangeKey2: Int | null;
  rangeKey3: Int | null;
  rangeKey4: Int | null;
  rangeKey5: Int | null;
  rangeKey6: Int | null;
  rangeKey7: Int | null;
  rangeKey8: Int | null;
  rangeKey9: Int | null;
  rangeKey10: Int | null;
  createdAt: MicroDate;
  updatedAt: MicroDate;
  expiresAt: MicroDate | null;
  attachments: StorageRecordAttachment[];
};

type WriteResult = {
  record: StorageRecord;
};

async write(
  countryCode: string,
  recordData: StorageRecordData,
  requestOptions: RequestOptions = {},
): Promise<WriteResult> {
  /* ... */
}

Below you can find the example of how to use the write method.

const recordData = {
  recordKey: '<key>',
  body: '<body>',
  profileKey: '<profile_key>',
  rangeKey1: 0,
  key2: '<key2>',
  key3: '<key3>'
}

const writeResult = await storage.write(countryCode, recordData);

List of available record fields

v3.0.0 release introduced a series of new fields available for data storage. Below you can find the full list of all the fields available for storage in InCountry Platform along with their types and storage methods. Each field is either encrypted, hashed or stored as follows:

String fields, hashed:

recordKey
parentKey
profileKey
serviceKey1
serviceKey2
serviceKey3
serviceKey4
serviceKey5

String fields, hashed if Storage options "hashSearchKeys" is set to true (by default it is):

WARNING If the hashSearchKeys option is set to false the following string fields will have length limitation of 256 characters at most.

key1
key2
key3
key4
key5
key6
key7
key8
key9
key10
key11
key12
key13
key14
key15
key16
key17
key18
key19
key20

String fields, encrypted:

body
precommitBody

Int fields, plain:

rangeKey1
rangeKey2
rangeKey3
rangeKey4
rangeKey5
rangeKey6
rangeKey7
rangeKey8
rangeKey9
rangeKey10

Date fields, plain:

createdAt
updatedAt
expiresAt

WARNING Records with non-null expiresAt value will be automatically deleted upon reaching the specified date.

Date fields

Use the createdAt and updatedAt fields to access date-related information about records. The createdAt field stores a date when a record was initially created in the target country. The updatedAt field stores a date of the latest write operation for the given recordKey.

Use expiresAt field to limit the lifespan of a record stored in InCountry Platform. The record will be automatically deleted upon reaching the specified date.

Below is an example of how to create a record that will be deleted in a day:

const nextDayDate = new Date();
nextDayDate.setDate(nextDayDate.getDate() + 1);

const recordData = {
  recordKey: '<key>',
  ...,
  expiresAt: nextDayDate,
}

const writeResult = await storage.write(countryCode, recordData);

NOTE

All dates are stored in UTC timezone and converted to UTC timezone in case of expiresAt.

SDK accepts ISO-8601 formatted strings, e.g '2021-03-11T17:23:05.941Z', numeric timestamps with decimal microseconds, Date and MicroDate objects. Microseconds are accepted using ISO string ('2021-03-11T17:23:05.941001Z'), float number timestamps (1630574845881.123) or MicroDate object.

It's highly recommended to provide timezone-aware dates to avoid any timezone-related issues in future.

Batches

You can use the batchWrite method to create/replace multiple records at once.

type BatchWriteResult = {
  records: Array<StorageRecord>;
};

async batchWrite(
  countryCode: string,
  records: Array<StorageRecordData>,
  requestOptions: RequestOptions = {},
): Promise<BatchWriteResult> {
  /* ... */
}

Example of usage:

batchResult = await storage.batchWrite(countryCode, recordDataArr);

Reading stored data

You can read the stored data records by its recordKey by using the read method. It accepts an object with the two fields - country and recordKey. It returns a Promise which is resolved to { record } or is rejected if there are no records for the passed recordKey.

type StorageRecord = {
  recordKey: string;
  body: string | null;
  parentKey: string | null;
  profileKey: string | null;
  precommitBody: string | null;
  key1?: string | null;
  key2?: string | null;
  key3?: string | null;
  key4?: string | null;
  key5?: string | null;
  key6?: string | null;
  key7?: string | null;
  key8?: string | null;
  key9?: string | null;
  key10?: string | null;
  key11?: string | null;
  key12?: string | null;
  key13?: string | null;
  key14?: string | null;
  key15?: string | null;
  key16?: string | null;
  key17?: string | null;
  key18?: string | null;
  key19?: string | null;
  key20?: string | null;
  serviceKey1: string | null;
  serviceKey2: string | null;
  serviceKey3: string | null;
  serviceKey4: string | null;
  serviceKey5: string | null;
  rangeKey1: Int | null;
  rangeKey2: Int | null;
  rangeKey3: Int | null;
  rangeKey4: Int | null;
  rangeKey5: Int | null;
  rangeKey6: Int | null;
  rangeKey7: Int | null;
  rangeKey8: Int | null;
  rangeKey9: Int | null;
  rangeKey10: Int | null;
  createdAt: MicroDate;
  updatedAt: MicroDate;
  expiresAt: MicroDate | null;
  attachments: StorageRecordAttachment[];
}

type ReadResult = {
  record: StorageRecord;
};

async read(
  countryCode: string,
  recordKey: string,
  requestOptions: RequestOptions = {},
): Promise<ReadResult> {
  /* ... */
}

Example of usage:

const readResult = await storage.read(countryCode, recordKey);

Find records

You can look up for data records either by using exact match search operators or partial text match operator in almost any combinations. More in Date fields section

type DateLike = string | number | Date | MicroDate;
type FilterDateQuery = DateLike | null | { $not?: DateLike | null; $gt?: DateLike; $gte?: DateLike; $lt?: DateLike; $lte?: DateLike; };
type FilterStrictDateQuery = DateLike | { $not?: DateLike; $gt?: DateLike; $gte?: DateLike; $lt?: DateLike; $lte?: DateLike; };
type FilterNumberQuery = number | number[] | null | { $not?: number | number[] | null; $gt?: number; $gte?: number; $lt?: number; $lte?: number; }; // number[] should be a non empty array
type FilterStringQuery = string | string[] | null | { $not?: string | string[] | null }; // string[] should be a non empty array
type FilterStringWithLikeQuery = string | string[] | null | { $not?: string | string[] | null, $like?: string };

type FindFilterStringFields = {
  recordKey: FilterStringQuery;
  parentKey: FilterStringQuery;
  key1: FilterStringWithLikeQuery;
  key2: FilterStringWithLikeQuery;
  key3: FilterStringWithLikeQuery;
  key4: FilterStringWithLikeQuery;
  key5: FilterStringWithLikeQuery;
  key6: FilterStringWithLikeQuery;
  key7: FilterStringWithLikeQuery;
  key8: FilterStringWithLikeQuery;
  key9: FilterStringWithLikeQuery;
  key10: FilterStringWithLikeQuery;
  key11: FilterStringWithLikeQuery;
  key12: FilterStringWithLikeQuery;
  key13: FilterStringWithLikeQuery;
  key14: FilterStringWithLikeQuery;
  key15: FilterStringWithLikeQuery;
  key16: FilterStringWithLikeQuery;
  key17: FilterStringWithLikeQuery;
  key18: FilterStringWithLikeQuery;
  key19: FilterStringWithLikeQuery;
  key20: FilterStringWithLikeQuery;
  profileKey: FilterStringQuery;
  serviceKey1: FilterStringQuery;
  serviceKey2: FilterStringQuery;
  serviceKey3: FilterStringQuery;
  serviceKey4: FilterStringQuery;
  serviceKey5: FilterStringQuery;
}

type FindFilterNumberFields = {
  rangeKey1: FilterNumberQuery;
  rangeKey2: FilterNumberQuery;
  rangeKey3: FilterNumberQuery;
  rangeKey4: FilterNumberQuery;
  rangeKey5: FilterNumberQuery;
  rangeKey6: FilterNumberQuery;
  rangeKey7: FilterNumberQuery;
  rangeKey8: FilterNumberQuery;
  rangeKey9: FilterNumberQuery;
  rangeKey10: FilterNumberQuery;
  version: FilterNumberQuery;
};

type FindFilterDateFields = {
  createdAt: FilterStrictDateLikeQuery;
  updatedAt: FilterStrictDateLikeQuery;
  expiresAt: FilterDateLikeQuery;
};

type FindFilter = Partial<FindFilterStringFields & FindFilterNumberFields & FindFilterDateFields {
  searchKeys: string;
  $or: Array<Partial<FindFilterStringFields>>
}>;

Exact match search

The following exact match search criteria are available:

• single value:

// Matches all records where record.key1 = 'abc' AND record.rangeKey1 = 1
{ key1: 'abc', rangeKey1: 1 }

• multiple values as an array:

// Matches all records where (record.key2 = 'def' OR record.key2 = 'jkl') AND (record.rangeKey1 = 1 OR record.rangeKey1 = 2)
{ key2: ['def', 'jkl'], rangeKey1: [1, 2] }

• a logical NOT operator for String fields and version:

// Matches all records where record.key3 <> 'abc'
{ key3: { $not: 'abc' } }

// Matches all records where record.key3 <> 'abc' AND record.key3 <> 'def'
{ key3: { $not: ['abc', 'def'] } }

// Matches all records where record.version <> 1
{ version: { $not: 1 }}

• comparison operators for Int fields:

// Matches all records where record.rangeKey1 >= 5 AND record.rangeKey1 <= 100
{ rangeKey1: { $gte: 5, $lte: 100 } }

• multiple criteria with $or operator:

{
  $or: [
    { key1: 'john', key2: 'smith' },
    { key1: 'smith', key2: 'john' },
  ]
}

{
  $or: [
    { key1: { $like: 'john' } },
    { key2: { $like: 'john' } },
  ]
}

Note: currently, nested $or operators and non-string fields inside $or operator are not supported.

Partial text match search

You can also look up for data records by partial match using the searchKeys operator which performs partial match search (similar to the LIKE SQL operator, without special characters) within records text fields key1, key2, ..., key20.

// Matches all records where record.key1 LIKE 'abc' OR record.key2 LIKE 'abc' OR ... OR record.key20 LIKE 'abc'
{ searchKeys: 'abc' }

Please note: The searchKeys operator cannot be used in combination with any of key1, key2, ..., key20 keys and works only in combination with the non-hashing Storage mode (hashSearchKeys param for Storage).

// Matches all records where (record.key1 LIKE 'abc' OR record.key2 LIKE 'abc' OR ... OR record.key20 LIKE 'abc') AND (record.rangeKey1 = 1 OR record.rangeKey1 = 2)
{ searchKeys: 'abc', rangeKey1: [1, 2] }

// Causes validation error (StorageClientError)
{ searchKeys: 'abc', key1: 'def' }

Another way to find records by partial key match is using $like operator. It provides a partial match search (similar to the LIKE SQL operator without special characters) against one of the record’s string fields key1, key2, ..., key20.

// Matches all records where record.key3 LIKE 'abc'
{ key3: { $like: 'abc' } }

Note: You can use either searchKeys or $like, not both.

Search options

The options parameter provides the following choices to manipulate the search results:

• limit allows to limit the total number of records returned;
• offset allows to specify the starting index used for records pagination;
• sort allows to sort the returned records by one or multiple keys;

WARNING To use sort in find() call for string keys key1...key20 you need to set hashSearchKeys option to false.

Sorting

Fields that records can be sorted by:

type SortKey =
  | 'createdAt'
  | 'updatedAt'
  | 'expiresAt'
  | 'key1'
  | 'key2'
  | 'key3'
  | 'key4'
  | 'key5'
  | 'key6'
  | 'key7'
  | 'key8'
  | 'key9'
  | 'key10'
  | 'key11'
  | 'key12'
  | 'key13'
  | 'key14'
  | 'key15'
  | 'key16'
  | 'key17'
  | 'key18'
  | 'key19'
  | 'key20'
  | 'rangeKey1'
  | 'rangeKey2'
  | 'rangeKey3'
  | 'rangeKey4'
  | 'rangeKey5'
  | 'rangeKey6'
  | 'rangeKey7'
  | 'rangeKey8'
  | 'rangeKey9'
  | 'rangeKey10';

Note: The SDK returns 100 records at most.

type SortItem = Partial<Record<SortKey, 'asc' | 'desc'>>; // each sort item should describe only one key!

type FindOptions = {
  limit?: number;
  offset?: number;
  sort?: NonEmptyArray<SortItem>;
};

type FindResult = {
  meta: {
    total: number;
    count: number;
    limit: number;
    offset: number;
  };
  records: Array<StorageRecord>;
  errors?: Array<{ error: StorageCryptoError; rawData: ApiRecord }>;
};

async find(
  countryCode: string,
  filter: FindFilter = {},
  options: FindOptions = {},
  requestOptions: RequestOptions = {},
): Promise<FindResult> {
  /* ... */
}

Example of usage

const filter = {
  key1: 'abc',
  key2: ['def', 'jkl'],
  key3: { $not: null },
  profileKey: 'test2',
  rangeKey1: { $gte: 5, $lte: 100 },
  rangeKey2: { $not: [0, 1] },
}

const options = {
  limit: 100,
  offset: 0,
  sort: [{ createdAt: 'asc' }, { rangeKey1: 'desc' }],
};

const findResult = await storage.find(countryCode, filter, options);

The returned findResult object looks like the following:

{
  records: [{/* StorageRecord */}],
  errors: [],
  meta: {
    limit: 100,
    offset: 0,
    total: 24
  }
}

with findResult.records sorted according to the following pseudo-sql:

SELECT * FROM record WHERE ...  ORDER BY createdAt asc, rangeKey1 desc

Find Error handling

You may encounter a situation when the find method receives records that cannot be decrypted. For example, this may happen once the encryption key has been changed while the found data was encrypted with the older version of that key. In such cases data returned by the find() method will be as follows:

{
  records: [/* successfully decrypted records */],
  errors: [/* errors */],
  meta: {/* ... */}
}: FindResult

Find one record

If you need to find only one of the records matching a specific filter, you can use the findOne method. If a record is not found, it returns { record: null }.

type FindOneResult = {
  record: StorageRecord | null;
};

async findOne(
  countryCode: string,
  filter: FindFilter = {},
  options: FindOptions = {},
  requestOptions: RequestOptions = {},
): Promise<FindOneResult> {
  /* ... */
}

Example of usage:

const findOneResult = await storage.findOne(countryCode, filter);

Deleting a single record

You can use the delete method to delete a record from InCountry Platform. It is possible by using the recordKey field only.

type DeleteResult = {
  success: true;
};

async delete(
  countryCode: string,
  recordKey: string,
  requestOptions: RequestOptions = {},
): Promise<DeleteResult> {
  /* ... */
}

Example of usage:

const deleteResult = await storage.delete(countryCode, recordKey);

Deleting multiple records

You can use the batchDelete method to delete multiple records from the InCountry platform. For now, the SDK allows deletion only by a list of recordKeys.

type DeleteFilter = {
  recordKey: string[];
};

type DeleteResult = {
  success: true;
};

async batchDelete(
  countryCode: string,
  filter: DeleteFilter,
  requestOptions: RequestOptions = {},
): Promise<DeleteResult> {
  /* ... */
}

Example of usage:

const deleteResult = await storage.batchDelete(countryCode, { recordKey: ['aaa'] });

Attaching files to a record

NOTE

Attachments are currently available for InCountry dedicated instances only. Please check your subscription plan for details. This may require specifying your dedicated instance endpoint when configuring NodeJS SDK Storage.

InCountry Storage allows you to attach files to the previously created records. Attachments' meta information is available through the attachments field of StorageRecord object.

type StorageRecord = {
  /* ... other fields ...  */
  attachments: StorageRecordAttachment[];
}

type StorageRecordAttachment = {
  fileId: string;
  fileName: string;
  hash: string;
  mimeType: string;
  size: number;
  createdAt: MicroDate;
  updatedAt: MicroDate;
  downloadLink: string;
}

Adding attachments

The addAttachment method allows you to add or replace attachments. File data can be provided either as Readable stream, Buffer or string with a path to the file in the file system.

Note: attaching file with size exceeding 100 Mb is not supported at the moment.

type AttachmentData = {
  file: Readable | Buffer | string;
  fileName: string;
}

async addAttachment(
  countryCode: string,
  recordKey: string,
  { file, fileName }: AttachmentData,
  upsert = false,
  requestOptions: RequestOptions = {},
): Promise<StorageRecordAttachment> {
  /* ... */
}

Example of usage:

// using file path
await storage.addAttachment(COUNTRY, recordData.recordKey, { file: '../file' });

// using data Stream
import * as fs from 'fs';

const file = fs.createReadStream('./LICENSE');
await storage.addAttachment(COUNTRY, recordData.recordKey, { file });

Deleting attachments

The deleteAttachment method allows you to delete attachment using its fileId.

async deleteAttachment(
  countryCode: string,
  recordKey: string,
  fileId: string,
  requestOptions: RequestOptions = {},
): Promise<unknown> {
  /* ... */
}

Example of usage:

await storage.deleteAttachment(COUNTRY, recordData.recordKey, attachmentMeta.fileId);

Downloading attachments

The getAttachmentFile method allows you to download attachment contents. It returns object with readable stream and filename.

async getAttachmentFile(
  countryCode: string,
  recordKey: string,
  fileId: string,
  requestOptions: RequestOptions = {},
): Promise<GetAttachmentFileResult> {
  /* ... */
}

Example of usage:

import * as fs from 'fs';

const { attachmentData } = await storage.getAttachmentFile(COUNTRY, recordData.recordKey, attachmentMeta.fileId);

const { file, fileName } = attachmentData;
const writeStream = fs.createWriteStream(`./${fileName}`);
file.pipe(writeStream);

Working with attachment meta info

The getAttachmentMeta method allows you to retrieve attachment's metadata using its fileId.

async getAttachmentMeta(
  countryCode: string,
  recordKey: string,
  fileId: string,
  requestOptions: RequestOptions = {},
): Promise<StorageRecordAttachment> {
  /* ... */
}

Example of usage:

const meta: StorageRecordAttachment = await storage.getAttachmentMeta(COUNTRY, recordData.recordKey, attachmentMeta.fileId);

The updateAttachmentMeta method allows you to update attachment's metadata (MIME type and file name).

type AttachmentWritableMeta = {
  fileName?: string;
  mimeType?: string;
};

async updateAttachmentMeta(
  countryCode: string,
  recordKey: string,
  fileId: string,
  fileMeta: AttachmentWritableMeta,
  requestOptions: RequestOptions = {},
): Promise<StorageRecordAttachment> {
    /* ... */
}

Example of usage:

await storage.updateAttachmentMeta(COUNTRY, data.recordKey, attachmentMeta.fileId, { fileName: 'new name!' });

Health Check

The healthcheck method of Storage allows you to check availability of a remote storage service by country code.

type HealthcheckResult = {
  result: boolean;
};

async healthcheck(
  countryCode: string,
  requestOptions: RequestOptions = {},
): Promise<HealthcheckResult> {
  /* ... */
}

Example of usage:

const { result } = await storage.healthcheck(COUNTRY);

Data Migration and Key Rotation support

Using getSecrets storage config options that provides SecretsData object enables key rotation and data migration support.

SDK introduces migrate method which allows you to re-encrypt data encrypted with old versions of the secret. It returns an object which contains some information about the migration - the amount of records migrated (migrated) and the amount of records left to migrate (totalLeft) (which basically means the amount of records with version different from currentVersion provided by SecretsData).

For a detailed example of a migration script please see examples/migration.js

type MigrateResult = {
  meta: {
    migrated: number;
    totalLeft: number;
  };
};

async migrate(
  countryCode: string,
  limit = FIND_LIMIT,
  findFilter: FindFilter = {},
  requestOptions: RequestOptions = {},
): Promise<MigrateResult> {
  /* ... */
}

Example of usage:

const migrateResult = await storage.migrate(countryCode);

AWS KMS integration

InCountry NodeJS SDK supports usage of any 32-byte (256-bit) AES key, including ones produced by AWS KMS symmetric master key (CMK).

The suggested use case assumes that AWS user already got his KMS encrypted data key (AES_256) generated. Afterwards the key gets decrypted using AWS Node.js client library (aws-sdk/clients/kms) and then provided to InCountry NodeJS SDK's getSecrets() function.

For a detailed example of AWS KMS keys usage please see examples/aws-kms.js

Error Handling

InCountry NodeJS SDK throws the following Exceptions:

• StorageConfigValidationError - used for Storage options validation errors. Can be thrown by any public method.

• SecretsProviderError - can be thrown during the call of getSecrets() function. Wraps the original error which occurred in getSecrets().

• SecretsValidationError - can be thrown if getSecrets() function returned secrets in wrong format.

• InputValidationError - used for input validation errors. Can be thrown by all public methods except Storage constructor.

• StorageAuthenticationError - can be thrown if SDK failed to authenticate in InCountry system with the provided credentials.

• StorageClientError - used for various errors related to Storage configuration. All of the errors classes StorageConfigValidationError, SecretsProviderError, SecretsValidationError, InputValidationError, StorageAuthenticationError are inherited from StorageClientError.

• StorageServerError - thrown if SDK failed to communicate with InCountry servers or if server response validation failed.

• StorageNetworkError - thrown if SDK failed to communicate with InCountry servers due to network issues, such as request timeout, unreachable endpoint etc. Inherited from StorageServerError.

• StorageCryptoError - thrown during encryption/decryption procedures (both default and custom). This may be a sign of malformed/corrupt data or a wrong encryption key provided to the SDK.

• StorageError - general exception. Inherited by all other exceptions

We suggest gracefully handling all the possible exceptions:

try {
  // use InCountry Storage instance here
} catch(e) {
  if (e instanceof StorageClientError) {
    // some input validation error

    // if you need to handle configuration errors more precisely:
    if (e instanceof StorageConfigValidationError) {
      // something is wrong with Storage options
    } else if (e instanceof SecretsProviderError) {
      // something is wrong with `getSecrets()` function. The original error is available in `e.data`
    } else if (e instanceof SecretsValidationError) {
      // something is wrong with `getSecrets()` function result
    } else if (e instanceof InputValidationError) {
      // something is wrong with input data passed to Storage public method
    } else if (e instanceof StorageAuthenticationError) {
      // something is wrong with the credentials provided in Storage options
    }
  } else if (e instanceof StorageServerError) {
    // some server error

    if (e instanceof StorageNetworkError) {
      // something is wrong with network connection
    } else {
      // server error or server response validation failed
    }
  } else if (e instanceof StorageCryptoError) {
    // some encryption error
  } else {
    // ...
  }
}

Custom encryption support

SDK supports the ability to provide custom encryption/decryption methods if you decide to use your own algorithm instead of the default one.

createStorage(options, customEncConfigs) allows you to pass an array of custom encryption configurations with the following schema, which enables custom encryption:

{
  encrypt: (text: string, secret: string | KeyObject, secretVersion: string) => Promise<string> | string,
  decrypt: (encryptedText: string, secret: string | KeyObject, secretVersion: string) => Promise<string> | string,
  isCurrent: boolean, // Optional but at most one in array should be isCurrent: true
  version: string
}

They should accept raw data to encrypt/decrypt, key data and key version received from getSecrets(). The resulted encrypted/decrypted data should be a string.

version attribute is used to differ one custom encryption from another and from the default encryption as well. This way SDK will be able to successfully decrypt any old data if encryption changes with time.

isCurrent attribute allows to specify one of the custom encryption configurations to use for encryption. Only one configuration can be set as isCurrent: true.

Here's an example of how you can set up SDK to use custom encryption (using XXTEA encryption algorithm):

const xxtea = require('xxtea');
const encrypt = (text, secret) => xxtea.encrypt(text, secret);
const decrypt = (encryptedText, secret) => xxtea.decrypt(encryptedText, secret);

const config = {
  encrypt,
  decrypt,
  isCurrent: true,
  version: 'current',
};

const getSecrets = () => {
  return {
    secrets: [
      {
        secret: 'longAndStrongPassword',
        version: 1,
        isForCustomEncryption: true
      }
    ],
    currentVersion: 1,
  };
}

const options = {
  environmentId: 'ENVIRONMENT_ID',
  oauth: {
    clientId: '<client_id>',
    clientSecret: '<client_secret>',
  },
  getSecrets: getSecrets,
};

const storage = await createStorage(options, [config]);

await storage.write('us', { recordKey: '<key>', body: '<body>' });

Testing Locally

1. In terminal run npm test for unit tests
2. In terminal run npm run integrations to run integration tests