dedupe-lambda v1.0.0

dedupe-lambda

This module allows you to ensure your lambda only fires exactly once! It requires a dynamoDB table that saves the lambda requestID (or any other unique identifier of your choice) and does a conditional put that saves the requestID if it doesn't exists or report a duplicate if it does exist. A TTL attribute is used to ensure dynamoDB automatically removes the entry after it expires.

Usage

const dedupe = require('./dedupe-lambda')

exports.handler = async (event, context) => {
  if (await dedupe(context)) return
  
  /*
    rest of your implementation
  */
}

Required dynamoDB table

This module requires a dynamoDB table with a primary partition key called key and a TTL attribute called ttl. By default, the function will look for a table called dedupe-lambda-table in the same region as the lambda. The name and region of the table can be configured as described in the Options section.

The CloudFormation properties for the table are available under dedupe-lambda/tableProperties.yml

Options

The dedupe function is to be called with the lambda event and/or a list of option overrides:

dedupe(context, {
  tableName: 'dedupe-test', // override table name
  tableRegion: 'eu-west-1', // override table region
  identifier: '124', // override unique identifier
  ttl: 10, // override ttl
})

All options have multiple ways of being defined (in order of increasing precedence):

• Unique identifier to use
  • from lambda event: awsRequestId
  • from overrides: identifier
• Name of the deduplication table
  • default name: dedupe-lambda-table
  • environment variable: $LAMBDA_DEDUPE_TABLE
  • from overrides: tableName
• Region of the deduplication table
  • from built-in lambda environment: $AWS_REGION
  • from environment: $LAMBDA_DEDUPE_TABLE_REGION
  • from overrides: tableRegion
• Time until the identifier expires (seconds)
  • default value: 890
  • from environment: $LAMBDA_DEDUPE_TTL
  • from overrides: ttl

Notes

In case of consuming an SQS queue, the messageId can be used. In this case, make sure the TTL is not higher than the message visibility timeout.