@manifest-cyber/job-common NPM

job-common

Utilties and functions shared across Manifest backend jobs.

Development

Run the integration tests:

yarn test:vitest:integration

or the unit tests:

yarn test:vitest:unit

Event Logging

Writes to jobLog collection on Mongo:

const thisJobLog = await jobLogWriter(null, '', 'started', 'job-osv-process', trigger);

Message Transports

When executing/calling from one job to another, you can use the logic patterns abstracted in either the messaging or job packages, depending on the granularity level needed.

I/O (`src/job` / `src/job/io`) - highest level of abstraction

If you want to enqueue job-sbom-process, then you can use:

// messageClient defined and loaded in scope
// env defined and loaded in scope

const sbomProcessQueue = getSbomProcessQueue(
  env,
  'sqs://example-dev-fallback-queue',
  'sqs://example-prod-fallback-queue',
);
const sbomRef = {
  mongoId: 'the-mongo-id',
  s3FileRef: {
    bucket: 'my-bucket',
    key: '<UUID>.json',
  },
};
enqueueSbomProcess(messageClient, sbomProcessQueue, sbomRef);

Message Client (`src/messaging`) - lower level abstraction

src/messaging contains Manifest's inter-service messaging client used across it's ecosystem, it currently supports sending and receiving messages via SQS or RabbitMQ through abstracted concepts.

Configuration

Configuration can be done directly via NewMessagingClient with the MessageClientConfiguration or it can be done from NewMessagingClientFromEnv which will delegate to the following environment variables:

Key	Description	Example
MFST_MSG_BROKER_URL	The broker URL of the connection	`https://sqs.us-east-1.amazonaws.com/123456`
MFST_MSG_QUEUE_TYPE	The type of queue for messaging (ie. `sqs`/`rabbitmq`)	`sqs`
MFST_MSG_CREATE_RESOURCES	Whether or not to create queues for messaging transmission	`true`
MFST_MSG_DELETE_WHEN_DONE	Whether ot not to delete messages after receipt and handling	`false`
MFST_MSG_INPUT_QUEUE_NAME	The queue to read from when receiving messages	`foo-bar-queue`
MFST_MSG_LOGGING	Whether or not to log message bodies received from payloads	`true`
MFST_MSG_TLS_ENABLE	Whether or not to enable TLS connections when connecting to the broker	`false`
MFST_MSG_TLS_CA_PATH	Filepath to CA (pem)	`/mnt/certs/ca.pem`
MFST_MSG_TLS_CERT_PATH	Filepath to TLS cert	`/mnt/certs/tls.pem`
MFST_MSG_TLS_CERT_PASSPHRASE	Passphrase for TLS cert	`supersecret`
MFST_MSG_TLS_KEY_PATH	Filepath to TLS key	`/mnt/certs/key.pem`

Sending/Receiving messages

Create a message handler to receive messages:

import { Message, MessageHandler } from '@manifest-cyber/job-common';

const messageHandler: MessageHandler = async (message: Message) => {
  try {
    // Check the message.Body, ie. JSON.parse(message.Body)
  } catch (error) {
    log.error('Some error occurred! Oh no!');
  }
};

Create the message client and start it

import { NewMessagingClientFromEnv } from '@manifest-cyber/job-common';

export const messageClient = NewMessagingClientFromEnv(messageHandler, log);
messageClient?.Start();

Assemble the message and send it:

let message = {
  MessageId: id,
  Body: body,
};

messagingClient.SendMessage(queue, message);

6 months ago

6 months ago

6 months ago

6 months ago

6 months ago

6 months ago

6 months ago

6 months ago

6 months ago

6 months ago

6 months ago

7 months ago

7 months ago

7 months ago

7 months ago

7 months ago

7 months ago

7 months ago

7 months ago

7 months ago

7 months ago

7 months ago

7 months ago

7 months ago

7 months ago

8 months ago

8 months ago

8 months ago

8 months ago

8 months ago

8 months ago

9 months ago

9 months ago

9 months ago

9 months ago

9 months ago

9 months ago

9 months ago

9 months ago

9 months ago

9 months ago

9 months ago

9 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

1.3.297-test-correct-1

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

10 months ago

11 months ago

11 months ago

11 months ago

11 months ago

11 months ago

11 months ago

11 months ago

11 months ago

11 months ago

11 months ago

11 months ago

11 months ago

11 months ago

12 months ago

12 months ago

12 months ago

12 months ago

12 months ago

12 months ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

job-common

Development

Event Logging

Message Transports

I/O (src/job / src/job/io) - highest level of abstraction

Message Client (src/messaging) - lower level abstraction

Configuration

Sending/Receiving messages

I/O (`src/job` / `src/job/io`) - highest level of abstraction

Message Client (`src/messaging`) - lower level abstraction