@keeex/sdk-helper

@keeex/sdk-helper

General description

Shared code to build SDK libraries.

Features:

• Simplified raw HTTP calls
• Authentication
• Handling of input values transformations into either JSON or FormData
• Handling of file inputs
• Handling of generic pagination values
• Handling of URI arguments and query arguments
• Handling of file replies
• Progress callback
• Concurrent request limitation
• Separate request queues

Call flow

This library implements a set of layers that are applied as needed to perform high-level API calls to low-level API calls.

High-level API calls are actual functions that can perform some logic, and are part of your SDK. These function have no particular constraint, and will at some point perform one or more mid-level API calls. This library provides functions to create these mid-level API calls. Their role is to convert a convenient input into something that fits into a low-level API call.

In turn, the low-level API call are actual, HTTP calls, with their usual constraints.

The following types are used throughout this README:

• (generic) InputType: the JavaScript input data, provided to the mid-level API call
• (generic) InputJson extends JSONObject: the JSON data format that is actually sent through the low-level API call (sometimes InputData is used for API data that can be something not JSON, like FormData)
• (generic) OutputJson extends JSONObject: the JSON data replied by the API call (sometimes OutputData can be used to include other reply types)
• (generic) OutputType: the JavaScript type returned by the mid-level API call
• (generic) RequestData: the final data passed to Axios (basically InputJson split between body, query, headers, etc.)
• (generic) ResponseData: the data retrieved from Axios (usually OutputJson)
• ApiFile: a structure used to pass files to the low-level API call.

The various steps applied to each mid-level API call are as follow:

• Caller provides input: InputType
• The input is processed through a conversion function, transforming InputType into InputJson (optional)
• Parameters are extracted from input: InputJson and placed where appropriate (body, query, params, headers, as defined in the mid-level API call definition) (body can be a JSON object, a multipart/form-data, or a raw file depending on the API call definition)
• If the route is authenticated, the registered authentication headers are added
• (hook) preAuthenticate() is called if the route is authenticated
• (global hook) preAuthenticate() is called if the route is authenticated
• (hook) preRequest() is called, and can alter parts of the request
• (global hook) preRequest() is called, and can alter parts of the request
• low-level API call is made, returning a raw axios object
• if the return code isn't a valid one, the error path is triggered (see below)
• (global hook) postRequest() is called with the raw Axios reply; it can alter it and return a different output: OutputJson object as data if needed
• (hook) postRequest() is called with output: OutputJson
• The output is processed through a conversion function, transforming OutputJson and file replies (if applicable) into OutputType
• The final output value is returned

Some variations can happen, for example pagination forces some structure into OutputJson, but that's the general flow.

The error path (if the HTTP status code is unexpected) change two things:

• if a local postError() hook is present, it is called and can trigger a retry once or "transform" the reply into a valid reply
• then, if a global postError() hook is present, it is called and can do the same (also only once per actual call, meaning that if both are present two retry can occur)
• if after all that, the error remain, an exception is raised before the postRequest() hooks.

Available call types

The following functions are available to create various kind of API calls:

Input \ Output	Nothing	Json	PaginatedJson	File
Nothing	noInNoOut	noInJsonOut	x	noInBinOut
Json	jsonInNoOut	jsonInJsonOut	paginatedJsonInJsonOut	jsonInBinOut
File	binInNoOut	binInJsonOut	x	binInBinOut

Each function have a variant named *Full that returns extra response data (headers and HTTP code) in addition to the converted data output.

In addition, the MidSdk class expose a way to get the configured Axios instance to perform custom requests manually.

Usage

Use the MidSdk class to create API mid-level calls, and use these in your own functions (or as-is, if applicable). During initialisation of your SDK, you can create the various mid-level API calls you need by providing a definition object.

Short example:

import {writeFileSync} from "node:fs";
import {MidSdk} from "@keeex/sdk-helper/midsdk.js"
import {AxiosConfigMode} from "@keeex/sdk-helper/types/axios.js";

/** Input provided to the `getFileInfo()` API call */
interface GetFileInfoRequest {
  id: number;
}

/** Output from the `getFileInfo()` API call */
interface GetFileInfoResponse {
  name: string;
  lastEditedAt: Date;
}

/** JSON output from the raw API call in `getFileInfo()` */
interface GetFileInfoResponseJson {
  name: string;
  lastEditedAt: string;
}

/** Input provided to the `getFile()` API call */
interface GetFileRequest {
  id: number;
}

export class MySdk extends MidSdk {
  // Definition of the `getFileInfo()` call
  public getFileInfo = this.jsonInJsonOut<
    GetFileInfoRequest,
    GetFileInfoResponse,
    GetFileInfoRequest,
    GetFileInfoResponseJson
  >({
    method: "GET",
    outputFilter: (rawResponse: GetFileInfoResponseJson): GetFileInfoResponse => ({
      ...rawResponse,
      lastEditedAt: new Date(rawResponse.lastEditedAt),
    }),
    route: "/file/:id/info",
  });

  // Definition of the `getFile()` call
  public getFile = this.jsonInBinOut<GetFileRequest>({
    method: "GET",
    route: "/file/:id",
  });

  public constructor() {
    super({axiosConfig: {baseUrl: "https://my.api.invalid", type: AxiosConfigMode.auto}});
  }
}

const sdk = new MySdk();
const fileInfo = await sdk.getFileInfo({id: 34});
const file = await sdk.getFile({id: 34});
writeFileSync(fileInfo.name, file.data);

The above example create an SDK that exposes two functions, getFileInfo() and getFile(). The first one returns a JSON object, the second one a file.

Global configuration

The constructor of MidSdk accepts a configuration object to set some global properties:

• authenticatedHeaders?: headers to use for authentication. This is the initial value and can be changed later on (see below).
• axiosConfig: Axios instance to use. This is mandatory.
• globalHooks?: Hooks that will be applied to all routes, unless explicitly excluded (see below).
• requestSettings?: Control some aspect of concurrent requests

Axios Config

The Axios config must match either of these interfaces:

/** Axios configuration to use an automatic instance */
interface AxiosAuto {
  /** URL prefix for all calls */
  baseUrl: string;
  /** Headers to add to all requests */
  headers?: Record<string, string>;
  type: AxiosConfigMode.auto;
}

/** Axios configuration to use a provided Axios instance */
interface AxiosCustom {
  /** Axios instance to use */
  axios: AxiosInstance;
  /**
   * URL prefix for all calls.
   * If provided, will override the `baseURL` of the provided `axios` object.
   */
  baseUrl?: string;
  /** Headers to add to all requests */
  headers?: Record<string, string>;
  type: AxiosConfigMode.custom;
}

The automatic version is easier and recommended. The AxiosCustom config is useful for testing purpose, when you use a custom Axios instance to access your service.

Request settings

If provided, this must match the following interface:

/** Parameters to control the processing of requests */
export interface RequestSettings {
  /** Limit the number of concurrent requests. Defaults to 2. */
  maxConcurrentRequests?: number;
  /** Optional proxy configuration */
  proxy?: ProxySettings;
}

/** Proxy configuration */
export interface ProxySettings {
  credentials?: {
    login: string;
    password: string;
  };
  /**
   * Proxy URL, in the form `<protocol>://<address>:<port>`
   *
   * Protocol can be "http", "https" or "socks"
   */
  url: string;
}

The maxConcurrentRequests is a value applied per Axios queue, and limit the number of concurrent requests on that queue.

Defining API routes

All API routes share the following properties.

Those are mandatory:

• method: HTTP method to use ("GET", "POST", etc.)
• route: the route path (relative to the base URL)

Those are optional:

• authentication?: if the route is authenticated, define the authentication method to use (see below)
• baseUrl?: override the globally set baseUrl for this route only
• hooks?: provide hooks that only applies to this route (see below)
• disableGlobalHooks?: toggle off global hooks for this route only
• queue?: name of the axios queue to use. All routes sharing the same queue will be limited in the number of active concurrent calls. If not provided, a default queue is created.
• responsePredicate?: a TypePredicate that applies to the raw reply returned by Axios. Usually used with JSON replies. When used in a paginated call, this predicate is applied to each returned row.
• validateStatus?: list of valid HTTP codes for this call. Defaults to accepting any 2XX status.

JSON input

API routes that takes a JSON as an input (the functions named jsonIn*()) accepts the following extra properties:

• headersParams?: properties to extract from the input into HTTP headers
• inputFilter?: function to convert the input into proper JSON for sending the request
• multipart?: force using a multipart/form-data
• queryParams?: keys to extract from the input into the query params

Additionally, if part of the route URL have path elements starting with :, they are replaced with input parameters. Any JSON property not used in the headers/query/route will be sent as the body.

JSON output

Calls that returns a JSON output can specify the property outputFilter? on their definition. This function is used to convert the raw JSON response from the request into the expected OutputType type.

File arguments

File arguments can be provided as top-level properties of a JSON input in different ways:

• a Blob
• a ApiFile object
• a FileDescriptionUri object (on React-Native only)

ApiFile objects have this interface:

/** Describe a file provided to an API call */
export interface ApiFile {
  data: Uint8Array;
  /** Defaults to `data.bin` */
  filename?: string;
  fileType: typeof apiFileType;
  /** Defaults to `application/octet-stream` */
  mimetype?: string;
}

FileDescriptionUri have this interface:

export interface FileDescriptionUri {
  /** File name sent. Defaults to "data.bin" */
  name?: string;
  /** Mimetype. Defaults to "application/octet-stream" */
  type?: string;
  /** URI for the host. */
  uri: string;
}

When provided as direct input (for functions named binIn*()), only ApiFile and Blob are supported.

Pagination

The function named paginationJsonInJsonOut() handles standard paginated query and reply. It sets the following properties in the input data:

• limit: maximum item per page
• order: requested sorting options
• page: page number

This function takes a generic type SortKeys that can be used to limit the available options for order.

In the reply, the following properties are available:

• limit: maximum item per page as set by the server
• order: the ordering used by the server
• page: current page number
• total: total number of items
• rows: array of elements

If any predicate is provided, it will be applied to the rows/sort keys. If no predicate at all are provided, no check will be done on the response.

Paginated requests have a few extra definition properties:

• extraPropsPredicate?: Type predicate for the extra top-level properties returned by the API call. This is used to validate the returned JSON.
• extraPropsFilter?: Convert the extra top-level properties not managed by the helper for a paginated reply. The general outputFilter property is applied to each row value in the reply and not to the top level. Use this property to convert top-level entries you want to extract from the raw response. Note that you have to provide this, as otherwise properties of the response will not be passed to the result automatically.
• sortKeysPredicate?: ensure that the value for the sorting key match the expected value. Default to matching all strings.

File replies

Routes returning a raw file will return an ApiFile instance.

Authentication

API definition can require an authentication mechanism. To do so, define authentication in their definition object to either true or a custom string. If enabled, it will add headers set in the MidSdk.authenticatedHeaders property matching the requested method.

The content of MidSdk.authenticatedHeaders can be setup on initialization, and changed at will during execution. In addition to the headers handling, the preAuthenticate() hook will be called so the request can be altered to suit custom needs.

Note that during error handling, it is possible to update MidSdk.authenticatedHeaders, but this will not be automatically applied on the immediate retry unless you return the value errorRetryAuth.

Hooks

There are 4 hooks available:

• preAuthenticate() called before a request is prepared and sent, to add authentication (if the route is authenticated)
• preRequest() to alter the raw request before execution
• postRequest() to alter the raw reply after execution
• postError() to handle errors during the request, and optionally allows retry or alter the result

Each of these hooks can be global, or defined for each route. When both (global and local) are defined, they are executed in the order defined in the beginning of this file.

Individual routes can prevent a global hook from executing.

All hooks operate on raw Axios data.

The postError() hook can be used to retry a request that failed, by returning the appropriate value (see PostErrorHook).

Progress callback

All mid-level API calls accept a progress parameter, that can be used to register progress callbacks.

The interface for progress handlers is:

export interface ProgressEvent {
  /** Completed data bytes */
  loaded: number;
  /** Total bytes */
  total?: number;
  /** Percentage completed (if available) */
  percent?: number;
  /** Number of bytes sent since the last event */
  sinceLastEvent: number;
  /** Estimated remaining time in seconds */
  estimated?: number;
  /** Speed in bytes/s */
  rate?: number;
}

export type ProgressHandler = (event: ProgressEvent) => void;

/** Provide progress handlers */
export interface ProgressHandlers {
  upload?: ProgressHandler;
  download?: ProgressHandler;
  /** Merge both upload and download as one handler */
  total?: ProgressHandler;
  /**
   * Balance ratio of upload/download for the unified progress event.
   *
   * A value of 0.5 means that upload will be 50% of the total progress.
   * A value of 0.9 means that upload will be 90% of the total progress.
   *
   * Defaults to 0.2.
   */
  totalRatio?: number;
}

The upload and download properties are explicit. The total handler is a custom handler that handles both the upload and download phases, and tries to estimate some values like remaining time and total progress based on the value of totalRatio.

Queue control

The default setting of this library is to not allow more than two concurrent requests on a single queue. However, to allow for long operations to coexist with short requests, it is possible to specify which "queue" is used by an API call. By putting a queue name on the queue property of an API call description, it can use a separate queue.

There is no restriction to the number of queues; keep in mind however that running too many concurrent requests can results in dropped replies in some circumstances.

If no queue name is provided a default queue is used.

Migrations

v7.x to v8.x

This is a big major release; most API are slightly changed. This section tries to outlines all the changes.

Very Breaking

• Old Proxy configuration not supported anymore (only the more recent format is valid)
• Hooks have been reworked, and are now very close to raw Axios processing, decoupled from the rest of request handling
• The filters mechanic is completely removed, replaced by two conversion function to be explicitely provided
• Internal files moved around. You should only rely on the barrel export.
• Old calls to define API functions are removed, and replaced with new calls that explicitly define which type of API call you're making. They must all be replaced.

Changed

• Configuration is split differently between request handling, proxy, axios, hooks, etc.
• File handling changed slightly; most cases should still work

v4.x to v5.x

Breaking

Removed deprecated calls

• createAPICall: use createApiCall
• directAPICall: use directApiCall

Rename to all uppercase acronym to pascal case one

• all API are renamed to Api (ex. toAPI becomes toApi)
• all SDK are renamed to Sdk (ex. SDKReply becomes SdkReply)
• all URL are renamed to Url (ex. baseURL becomes baseUrl)
• all CB are renamed to Cb (ex. progressCB becomes progressCb)
• all AB are renamed to Ab (ex. fromApiAB becomes fromApiAb)