1.4.0 • Published 4 months ago

@adobe/asset-compute-commons v1.4.0

Weekly downloads
557
License
Apache-2.0
Repository
github
Last release
4 months ago

Version

asset-compute-commons

Common utilities needed by all Asset Compute serverless actions

Installation

npm install @adobe/asset-compute-commons

API Details

Asset Compute commons contains many different libraries that take care of common functionalities for workers like Adobe IO Events, New Relic metrics, and custom errors.

Asset Compute Events

Asset Compute event handler for sending Adobe IO Events

Constructor parameters

AssetComputeEvents constructor supports the following mandatory parameters:

  • params: object must contain authorization parameters: - auth.accessToken - orgId or auth.orgId - clientId or auth.clientId

AssetComputeEvents constructor supports the following optional parameters:

  • retry: retry options from node-fetch-retry for sending IO events. (defaults to node-fetch-retry default options)

Sending Events

The sendEvent method sends an Adobe IO Event.

AssetComputeEvents.sendEvent method takes in two required parameters: type and payload.

Examples

Example with custom retry options:

const { AssetComputeEvents } = require('@adobe/asset-compute-commons');

const params = {
	auth: {
		accessToken: '12345',
		orgId: 'orgId',
		clientId: 'clientId`
	}
}
const retry = {
	retryMaxDuration: 1000 // in ms
}
const eventsHandler  = new AssetComputeEvents(params, retry);

await eventsHandler.sendEvent('rendition_created', {
            rendition: {
				name: 'rendition.jpg',
				fmt: 'jpg'
			}
      });

Example with a rendition_failed event type:

const { AssetComputeEvents } = require('@adobe/asset-compute-commons');

const params = {
	auth: {
		accessToken: '12345',
		orgId: 'orgId',
		clientId: 'clientId`
	}
}
const eventsHandler  = new AssetComputeEvents(params);

await eventsHandler.sendEvent('rendition_failed', {
            rendition: {
				name: 'rendition.21',
				fmt: '21'
			},
			errorReason: 'RenditionFormatUnsupported',
			errorMessage: 'Rendition format `21` is not supported'
     });

Asset Compute Metrics

Asset Compute metrics handler for sending New Relic metrics. It uses a node js agent node-openwhisk-newrelic to send metrics to New Relic.

Constructor parameters

AssetComputeMetrics constructor supports the following mandatory parameters:

  • params: Object must contain New Relic metrics parameters: - newRelicEventsURL: New Relic Insights Events url (should be of the form: https://insights-collector.newrelic.com/v1/accounts/<YOUR_ACOUNT_ID>/events) - newRelicApiKey: New Relic Insights API key (see the "Register an Insert API key" section here)

AssetComputeMetrics constructor supports the following optional parameters:

Simple example

Initiates metrics handler, sends metrics and stops metrics agent:

const { AssetComputeMetrics } = require('@adobe/asset-compute-commons');

const params = {
	newRelicEventsURL: 'https://insights-collector.newrelic.com/v1/accounts/<YOUR_ACOUNT_ID>/events',
	newRelicApiKey: 'YOUR_API_KEY',
	// ... rest of the Asset Compute parameters

}
const metricsHandler = new AssetComputeMetrics(params);
const metrics = {
	downloadDuration: 200,
	size: 3000
}
await metricsHandler.sendMetrics('rendition', metrics);
await metricsHandler.activationFinished(); // see https://github.com/adobe/node-openwhisk-newrelic#usage for information about `activationFinished()`

Other additional functions

Adding custom metrics:

metricsHandler.add({
	uuid: '12345',
	count: 2
});

Get current state of metrics:

console.log(metricsHandler.get()); // should print out metrics added in `metricsHandler.add()`

Sending error metrics (sends metrics with event type error):

const location = 'upload_worker_flite';
const message = 'Invalid file format';
const metrics = {
	downloadDuration: 200,
	size: 3000
}
await metricsHandler.sendErrorMetrics(location, message, metrics);

Sending client error metrics (sends metrics with event type client_error):

const location = 'upload_worker_flite';
const message = 'Invalid file format';
const metrics = {
	downloadDuration: 200,
	size: 3000
}
await metricsHandler.sendClientErrorMetrics(location, message, metrics);

Sending error metrics by exceptions thrown (sends metrics with event type client_error or error depending on the error):

foo() {
	try {
		console.log('hello!');
	} catch (error) {
		metricsHandler.handleError(error, {
			location: "mylocation",
			message: "something failed",
			metrics: {}
		});
	}
}

Asset Compute Errors

There are several custom errors used in Asset Compute workers:

Error types

TypeDescriptionProperties
errorunexpected errors and exceptionsmessage, date, location
client_errorerrors caused by client misconfigurationmessage, date, reason

Error Properties

  • message: error message
  • date: current time in UTC of when the error was thrown
  • location: location where the error took place (only in type error)
  • reason: the reason for the error (only in client errors) - must be one of list: "SourceFormatUnsupported", "RenditionFormatUnsupported", "SourceUnsupported", "SourceCorrupt", "RenditionTooLarge" }

Custom Errors

NameDescriptionType
SourceFormatUnsupportedErrorThe source is of an unsupported type.client error
RenditionFormatUnsupportedErrorThe requested format is unsupported.client error
SourceUnsupportedErrorThe specific source is unsupported even though the type is supported.client error
SourceCorruptErrorThe source data is corrupt. Includes empty files.client error
RenditionTooLargeErrorThe rendition could not be uploaded using the pre-signed URL(s) provided in target. The actual rendition size is available as metadata in repo:size and can be used by the client to re-process this rendition with the right number of pre-signed URLs.client error
ArgumentErrorWrong arguments (type, structure, etc.)error
GenericErrorAny other error.error

Examples

Generic error downloading the source file in action worker-pie:

const { GenericError } = require('@adobe/asset-compute-commons');

const message = 'Error while downloading source file!'
const location = 'download_worker_pie'
throw new GenericError(message, location);

Rendition format is unsupported:

const { RenditionFormatUnsupportedError } = require('@adobe/asset-compute-commons');

const message = 'Rendition format `sdfg` is not supported'
throw new RenditionFormatUnsupportedError(message);

Asset Compute Log Utils

Utilities for removing sensitive information from Asset Compute worker logs

Examples

Redacting access token from logs:

const { AssetComputeLogUtils } = require('@adobe/asset-compute-commons');

params = {
	accessToken: '123453467',
	fmt: 200
}
console.log("Asset Compute parameters:", AssetComputeLogUtils.redactUrl(params)); // should replace access token with "[...REDACTED...]"

Prints out exact same logs using AssetComputeLogUtils.log method:

const { AssetComputeLogUtils } = require('@adobe/asset-compute-commons');

params = {
	accessToken: '123453467',
	fmt: 200
}
AssetComputeLogUtils.log(params, "Asset Compute parameters"); // should replace access token with "[...REDACTED...]"

Apache OpenWhisk Action Name

A simple way to get information about the Apache OpenWhisk action.

Properties (all default to empty strings)

  • name: base Apache OpenWhisk action name
  • package: Apache OpenWhisk package name
  • namespace: Apache OpenWhisk namespace
  • fullname: full Apache OpenWhisk action name, including namespace, package and action name from environment variable __OW_ACTION_NAME

Examples:

const actionInfo = new OpenwhiskActionName();
console.log(actionInfo.name) // prints out something like `worker-pie`
console.log(actionInfo.package) // prints package name, ex: `experimental`
console.log(actionInfo.namespace) // prints namespace, ex: `stage`
console.log(actionInfo.fullname) // prints full name, ex: /stage/experimental/worker-pie

Asynchronous Events

When processing is finished, or if errors occurred, events are sent through Adobe I/O Events. Events are JSON objects in the event field of the objects in the events array of the jorunal response.

The I/O event type for all events of the Asset Compute service is asset_compute. The journal will be automatically subscribed to this event type only, and consumers are not expected to filter based on the i/o event type.

The service specific event types are available in the type property of the event.

Event Types

EventDescription
rendition_createdSent for each successfully processed and uploaded rendition.
rendition_failedSent for each rendition that failed to process or upload.

Event Attributes

AttributeTypeEventDescription
datestring*Timestamp when event was sent in simplified extended ISO-8601 format (as defined by Javascript Date.toISOString()).
requestIdstring*The request id of the original request to /process, same as X-Request-Id header.
sourceobject*The source of the /process request.
userDataobject*The userData of the /process request if set.
renditionobjectrendition_*The corresponding rendition object passed in /process.
metadataobjectrendition_createdThe metadata properties of the rendition.
errorReasonstringrendition_*Rendition failure reason if any.
errorMessagestringrendition_*Text giving more detail about the rendition failure if any.

Metadata

PropertyDescription
tiff:ImageWidthThe width of the rendition in pixels. Will not be present if the rendition is not an image.
tiff:ImageLengthThe length of the rendition in pixels. Will not be present if the rendition is not an image.
repo:sizeThe size of the rendition in bytes.
repo:sha1The sha1 digest of the rendition.

Error Reasons

ReasonDescription
SourceFormatUnsupportedThe source is of an unsupported type.
RenditionFormatUnsupportedThe requested format is unsupported.
SourceUnsupportedThe specific source is unsupported even though the type is supported.
SourceCorruptThe source data is corrupt. Includes empty files.
RenditionTooLargeThe rendition could not be uploaded using the pre-signed URL(s) provided in target. The actual rendition size is available as metadata in repo:size and can be used by the client to re-process this rendition with the right number of pre-signed URLs.
GenericErrorAny other error.

Contributing

Contributions are welcomed! Read the Contributing Guide for more information.

Licensing

This project is licensed under the Apache V2 License. See LICENSE for more information.


1.4.0

4 months ago

1.3.1

4 months ago

1.3.0

5 months ago

1.2.1

5 months ago

1.2.0

1 year ago

1.1.9

2 years ago

1.1.10

2 years ago

1.1.8

2 years ago

1.1.7

2 years ago

1.1.6

2 years ago

1.1.5

2 years ago

1.1.4

2 years ago

1.1.3

3 years ago

1.1.2

4 years ago

1.1.1

4 years ago

1.1.0

4 years ago

1.0.1

4 years ago

1.0.0

4 years ago