@handy-common-utils/aws-utils v2.11.0

@handy-common-utils/aws-utils

AWS related utilities that are compatible with both AWS Javascript SDK v3 and v2, and also some utilities that require v3.

How to use

Core

First add it as a dependency:

npm install @handy-common-utils/aws-utils

Then you can use it in the code:

import { AwsUtils } from '@handy-common-utils/aws-utils';

const domainNameObjects = await AwsUtils.repeatFetchingItemsByPosition(
  pagingParam => apig.getDomainNames({ limit: 100, ...pagingParam }).promise(),
);

You can either import and use the class as shown above, or you can import individual functions directly like below:

import { repeatFetchingItemsByNextToken, repeatFetchingItemsByMarker, parseArn } from '@handy-common-utils/aws-utils';

S3

S3 related utility functions can be imported and used in this way:

import { S3Client } from '@aws-sdk/client-s3';
import { decodeS3ObjectKey, deleteS3Object } from '@handy-common-utils/aws-utils/s3';

const srcEncodedKey = record.s3.object.key;
const srcKey = decodeS3ObjectKey(srcEncodedKey);
const destKey = srcKey.replace('/src-dir/', '/dest-dir/')

const s3 = new S3Client();
await copyS3Object(s3, bucket, srcEncodedKey, destKey);
await deleteS3Object(s3, bucket, srcKey);

To use S3 related utilities, you need to add @aws-sdk/client-s3 as a dependency of your project because it is not included as a dependency of this package.

SSM

SSM related utility functions can be imported and used in this way:

import { SSM } from '@aws-sdk/client-ssm';
import { getSsmParameter, getSsmParameterParsed } from '@handy-common-utils/aws-utils/ssm';

const ssm = new SSM();
const workDir = await getSsmParameter(ssm, '/my-config/work-dir', '/tmp');
const config = await getSsmParameterParsed<Config>(ssm, '/my-config/config');

To use SSM related utilities, you need to add @aws-sdk/client-ssm as a dependency of your project because it is not included as a dependency of this package.

API

@handy-common-utils/aws-utils

Modules

• aws-utils
• s3
• ssm

Classes

Class: AwsUtils

aws-utils.AwsUtils

Constructors

constructor

• new AwsUtils()

Methods

dynamodbLocalClientOptions

▸ Static dynamodbLocalClientOptions(endpoint?): Object

Build an object that can be passed into DynamoDB.DocumentClient(...) for DynamoDB Local (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html)

Parameters

Returns

Object

the options object

Example

const ddbClient = new DynamoDB.DocumentClient(process.env.IS_OFFLINE === 'true' ? AwsUtils.dynamodbLocalClientOptions() : undefined);

fetchAllByContinuationToken

▸ Static fetchAllByContinuationToken\<T, M>(fetchItemsByContinuationToken, itemsFieldName?): Promise\<T[]>

Fetch all items through repeatedly calling API with ContinuationToken/NextContinuationToken based pagination. This function is useful for client side pagination when the response from AWS API contains NextContinuationToken and items fields.

Type parameters

Parameters

Returns

Promise\<T[]>

all items fetched

Example

const objects = await fetchAllByContinuationToken(() => s3.send(new ListObjectsV2Command({Bucket: bucket})));

fetchAllByExclusiveStartKey

▸ Static fetchAllByExclusiveStartKey\<T, K>(fetchItemsByExclusiveStartKey, itemsFieldName?): Promise\<T[]>

Fetch all items through repeatedly calling API with ExclusiveStartKey/LastEvaluatedKey based pagination. This function is useful for client side pagination when the response from AWS API contains LastEvaluatedKey and items fields.

Type parameters

Parameters

Returns

Promise\<T[]>

all items fetched

Example

const allItemsInDynamoDbTable = await AwsUtils.fetchAllByExclusiveStartKey<MyTableItem>(
  pagingParam => dynamoDbDocumentClient.scan({...pagingParam, TableName: 'my-table', limit: 20}).promise(),
);

fetchAllByMarker

▸ Static fetchAllByMarker\<T, M>(fetchItemsByMarker, itemsFieldName): Promise\<T[]>

Fetch all items through repeatedly calling API with Marker/NextMarker based pagination. This function is useful for client side pagination when the response from AWS API contains NextMarker and items fields.

Type parameters

Parameters

Returns

Promise\<T[]>

all items fetched

Example

const functionConfigurations = await AwsUtils.fetchAllByMarker<Lambda.FunctionConfiguration>(
  pagingParam => withRetry(() => lambda.listFunctions({ ...pagingParam }).promise()),
  'Functions',
);

fetchAllByNextToken

▸ Static fetchAllByNextToken\<T, K>(fetchItemsByNextToken, itemsFieldName): Promise\<T[]>

Fetch all items through repeatedly calling API with NextToken based pagination. This function is useful for client side pagination when the response from AWS API contains NextToken and items fields.

Type parameters

Parameters

Returns

Promise\<T[]>

all items fetched

Example

const topics = await AwsUtils.fetchAllByNextToken<SNS.Topic>(
  pagingParam => sns.listTopics({...pagingParam}).promise(),
  'Topics',
);

fetchAllByNextTokenV3

▸ Static fetchAllByNextTokenV3\<T, K>(fetchItemsByNextToken, itemsFieldName): Promise\<T[]>

Fetch all items through repeatedly calling API with nextToken based pagination which is used in aws-sdk v3. This function is useful for client side pagination when the response from AWS API contains nextToken and items fields.

Type parameters

Parameters

Returns

Promise\<T[]>

all items fetched

Example

const executions = await AwsUtils.fetchAllByNextTokenV3<ExecutionListItem>(
  (pagingParam) => this.client.send(new ListExecutionsCommand({
    stateMachineArn,
    statusFilter: status,
    ...pagingParam,
  })),
  'executions',
);

fetchAllByPosition

▸ Static fetchAllByPosition\<T, P>(fetchItemsByPosition, itemsFieldName?): Promise\<T[]>

Fetch all items through repeatedly calling API with position based pagination. This function is useful for client side pagination when the response from AWS API contains position and items fields.

Type parameters

Parameters

Returns

Promise\<T[]>

all items fetched

Example

const domainNameObjects = await AwsUtils.fetchingAllByPosition(
  pagingParam => apig.getDomainNames({limit: 500, ...pagingParam}).promise(),
);

fetchAllWithPagination

▸ Static fetchAllWithPagination\<IT, RT, IFN, PFN, PFT>(fetchOnePageOfItems, itemsFieldName, paginationFieldName, shouldFetchNextPage?): Promise\<Exclude\<RTIFN, undefined>>

Fetch all items through repeatedly calling pagination based API. This function is useful for client side pagination when the calling AWS API.

Type parameters

Parameters

Returns

Promise\<Exclude\<RTIFN, undefined>>

all items fetched

Example

const executions = await AwsUtils.fetchAllWithPagination(
  (pagingParam) => this.client.send(new ListExecutionsCommand({
    stateMachineArn,
    statusFilter: status,
    ...pagingParam,
  })),
  'executions',
  'nextToken',
  (resp) => resp.executions != null && resp.executions.some((exe) => exe.startDate!.getTime() <= beginTime),
);

fibonacciRetryConfigurationOptions

▸ Static fibonacciRetryConfigurationOptions(maxRetries, base?): PartialConfigurationOptions

Generate part of a ConfigurationOptions object having maxRetries as specified and a custom RetryDelayOptions for fibonacci sequence based retry delays.

Parameters

Returns

PartialConfigurationOptions

part of a ConfigurationOptions object that has maxRetries as specified and a customBackoff utilising fibonacci sequence for calculating delays

parseArn

▸ Static parseArn(arn): undefined | null | Arn & { arn: string }

Parse ARN

Parameters

Returns

undefined | null | Arn & { arn: string }

null or undefined if the input is null or undefined, or parsed ARN including the original ARN string

promiseWithRetry

▸ Static promiseWithRetry\<Result, TError>(operation, backoff?, statusCodes?): Promise\<Result>

Perform an AWS operation (returning a Request) with retry. This function is quite handy when you are using AWS SDK v2. If you are using AWS SDK v3, use withRetry(...) instead.

The retry could happen only if the error coming from AWS has property retryable/$retryable equals to true. If you don't want retryable/$retryable property to be checked, use PromiseUtils.withRetry(...) directly.

Type parameters

Parameters

Returns

Promise\<Result>

result came out from the last attempt

withRetry

▸ Static withRetry\<Result, TError>(operation, backoff?, statusCodes?): Promise\<Result>

Perform an AWS operation (returning a Promise) with retry. This function is quite handy when you are using AWS SDK v3. If you are using AWS SDK v2, promiseWithRetry(...) could be more convenient.

The retry could happen only if the error coming from AWS has property retryable/$retryable equals to true. If you don't want retryable/$retryable property to be checked, use PromiseUtils.withRetry(...) directly.

Type parameters

Parameters

Returns

Promise\<Result>

result came out from the last attempt

See

promiseWithRetry

Interfaces

Interface: S3ObjectSummary

s3.S3ObjectSummary

Properties

Module: aws-utils

Re-exports

Functions

• fetchAllByNextToken = AwsUtils.fetchAllByNextToken
• fetchAllByNextTokenV3 = AwsUtils.fetchAllByNextTokenV3
• fetchAllWithPagination = AwsUtils.fetchAllWithPagination
• fetchAllByMarker = AwsUtils.fetchAllByMarker
• fetchAllByExclusiveStartKey = AwsUtils.fetchAllByExclusiveStartKey
• withRetry = AwsUtils.withRetry
• promiseWithRetry = AwsUtils.promiseWithRetry
• fibonacciRetryConfigurationOptions = AwsUtils.fibonacciRetryConfigurationOptions
• parseArn = AwsUtils.parseArn
• dynamodbLocalClientOptions = AwsUtils.dynamodbLocalClientOptions

Exports

Classes

• AwsUtils

Type Aliases

PossibleAwsError

Ƭ PossibleAwsError: Object

An error that is possibly generated by AWS SDK v2 or v3

Type declaration

Variables

FIBONACCI_SEQUENCE_BACKOFFS

• Const FIBONACCI_SEQUENCE_BACKOFFS: number[]

Module: s3

Interfaces

• S3ObjectSummary

Functions

copyS3Object

▸ copyS3Object(s3, srcBucket, srcEncodedKey, destDecodedKey, metadata?, destBucket?): Promise\<CopyObjectCommandOutput>

Copy S3 object

Parameters

Returns

Promise\<CopyObjectCommandOutput>

S3 command output

decodeS3ObjectKey

▸ decodeS3ObjectKey(key): string

Decode the raw object key which is URL encoded and could contain "+" as replacement of " "

Parameters

Returns

string

Decoded key

deleteS3Object

▸ deleteS3Object(s3, bucket, key): Promise\<DeleteObjectCommandOutput>

Delete an S3 object. No error would be thrown if the object does not exist.

Parameters

Returns

Promise\<DeleteObjectCommandOutput>

S3 command output

encodeS3ObjectKey

▸ encodeS3ObjectKey(key): string

URL encode the object key, and also replace "%20" with " " and "%2F with "/" which is the convention of AWS

Parameters

Returns

string

URL encoded object key

generatePresignedUrlForDownloading

▸ generatePresignedUrlForDownloading(s3, bucket, key, expiresIn, options?): Promise\<string>

Generate a pre-signed URL for downloading the S3 object

Parameters

Returns

Promise\<string>

An URL that can be used to download the S3 object.

generatePresignedUrlForUploading

▸ generatePresignedUrlForUploading(s3, bucket, key, expiresIn): Promise\<string>

Generate a pre-signed URL for uploading content to the S3 object

Parameters

Returns

Promise\<string>

An URL that can be used to upload content to the S3 object.

getS3ObjectContentByteArray

▸ getS3ObjectContentByteArray(s3, bucket, key, range?): Promise\<Uint8Array | undefined>

Get the content of the S3 object as a Uint8Array.

Parameters

Returns

Promise\<Uint8Array | undefined>

Content of the S3 object as a Uint8Array. If the object does not have content, an empty Uint8Array will be returned. If the object does not exist, undefined will be returned.

getS3ObjectContentString

▸ getS3ObjectContentString(s3, bucket, key, encoding?): Promise\<string | undefined>

Get the content of the S3 object as a string.

Parameters

Returns

Promise\<string | undefined>

Content of the S3 object as a string. If the object does not have content, an empty string will be returned. If the object does not exist, undefined will be returned.

headS3Object

▸ headS3Object(s3, bucket, key): Promise\<HeadObjectCommandOutput | undefined>

Get details of the S3 object without downloading its content.

Parameters

Returns

Promise\<HeadObjectCommandOutput | undefined>

S3 command output, or undefined if the object does not exist.

putS3Object

▸ putS3Object(s3, bucket, key, content, options?): Promise\<PutObjectOutput>

Store content into S3.

Parameters

Returns

Promise\<PutObjectOutput>

PutObjectOutput

scanS3Bucket

▸ scanS3Bucket(s3, bucket, options?): Promise\<S3ObjectSummary[]>

Scan S3 bucket and return both normal objects and directory objects. Directory objects have keys ending with '/'. This function handles pagination automatically.

Parameters

Returns

Promise\<S3ObjectSummary[]>

Array of normal and directory objects found

Module: ssm

Functions

getSsmParameter

▸ getSsmParameter(ssm, parameterName, fallbackValue?): Promise\<string | undefined>

Get a parameter from SSM Parameter Store. If the parameter is not found and a fallbackValue is given, the fallbackValue will be returned. If the parameter is not found and there is no fallbackValue given, undefined will be returned. If any other error occurs, the error will be thrown.

Parameters

Returns

Promise\<string | undefined>

Value of the parameter, or the fallbackValue, or undefined

getSsmParameterParsed

▸ getSsmParameterParsed\<T>(ssm, parameterName, fallbackValue?): Promise\<T | undefined>

Get a parameter from SSM Parameter Store and return it parsed as JSON. If the parameter is not found and a fallbackValue is given, the fallbackValue will be returned. If the parameter is not found and there is no fallbackValue given, undefined will be returned. If any other error occurs, the error will be thrown.

Type parameters

Parameters

Returns

Promise\<T | undefined>

Value of the parameter parsed as JSON, or the fallbackValue, or undefined