1.3.395 • Published 9 months ago

@manifest-cyber/job-common v1.3.395

Weekly downloads
-
License
UNLICENSED
Repository
github
Last release
9 months ago

job-common

Utilties and functions shared across Manifest backend jobs.

Development

Run the integration tests:

yarn test:vitest:integration

or the unit tests:

yarn test:vitest:unit

Event Logging

Writes to jobLog collection on Mongo:

const thisJobLog = await jobLogWriter(null, '', 'started', 'job-osv-process', trigger);

Message Transports

When executing/calling from one job to another, you can use the logic patterns abstracted in either the messaging or job packages, depending on the granularity level needed.

I/O (src/job / src/job/io) - highest level of abstraction

If you want to enqueue job-sbom-process, then you can use:

// messageClient defined and loaded in scope
// env defined and loaded in scope

const sbomProcessQueue = getSbomProcessQueue(
  env,
  'sqs://example-dev-fallback-queue',
  'sqs://example-prod-fallback-queue',
);
const sbomRef = {
  mongoId: 'the-mongo-id',
  s3FileRef: {
    bucket: 'my-bucket',
    key: '<UUID>.json',
  },
};
enqueueSbomProcess(messageClient, sbomProcessQueue, sbomRef);

Message Client (src/messaging) - lower level abstraction

src/messaging contains Manifest's inter-service messaging client used across it's ecosystem, it currently supports sending and receiving messages via SQS or RabbitMQ through abstracted concepts.

Configuration

Configuration can be done directly via NewMessagingClient with the MessageClientConfiguration or it can be done from NewMessagingClientFromEnv which will delegate to the following environment variables:

KeyDescriptionExample
MFST_MSG_BROKER_URLThe broker URL of the connectionhttps://sqs.us-east-1.amazonaws.com/123456
MFST_MSG_QUEUE_TYPEThe type of queue for messaging (ie. sqs/rabbitmq)sqs
MFST_MSG_CREATE_RESOURCESWhether or not to create queues for messaging transmissiontrue
MFST_MSG_DELETE_WHEN_DONEWhether ot not to delete messages after receipt and handlingfalse
MFST_MSG_INPUT_QUEUE_NAMEThe queue to read from when receiving messagesfoo-bar-queue
MFST_MSG_LOGGINGWhether or not to log message bodies received from payloadstrue
MFST_MSG_TLS_ENABLEWhether or not to enable TLS connections when connecting to the brokerfalse
MFST_MSG_TLS_CA_PATHFilepath to CA (pem)/mnt/certs/ca.pem
MFST_MSG_TLS_CERT_PATHFilepath to TLS cert/mnt/certs/tls.pem
MFST_MSG_TLS_CERT_PASSPHRASEPassphrase for TLS certsupersecret
MFST_MSG_TLS_KEY_PATHFilepath to TLS key/mnt/certs/key.pem

Sending/Receiving messages

  1. Create a message handler to receive messages:
import { Message, MessageHandler } from '@manifest-cyber/job-common';

const messageHandler: MessageHandler = async (message: Message) => {
  try {
    // Check the message.Body, ie. JSON.parse(message.Body)
  } catch (error) {
    log.error('Some error occurred! Oh no!');
  }
};
  1. Create the message client and start it
import { NewMessagingClientFromEnv } from '@manifest-cyber/job-common';

export const messageClient = NewMessagingClientFromEnv(messageHandler, log);
messageClient?.Start();
  1. Assemble the message and send it:
let message = {
  MessageId: id,
  Body: body,
};

messagingClient.SendMessage(queue, message);
@aws-sdk/client-s3@aws-sdk/client-secrets-manager@aws-sdk/client-sqs@aws-sdk/s3-request-presigner@huggingface/hub@manifest-cyber/db-connect@manifest-cyber/logger-ts@smithy/util-utf8@types/expressae-cvss-calculatoramqpamqplibdotenvfast-xml-parserjsonwebtokenluxonopenaipackageurl-jsrabbitmq-clientsemversqs-consumersqs-producerundiciwinstonwinston-daily-rotate-file
1.3.395

9 months ago

1.3.393

9 months ago

1.3.391

9 months ago

1.3.389

9 months ago

1.3.387

9 months ago

1.3.385

9 months ago

1.3.383

9 months ago

1.3.379

9 months ago

1.3.376

9 months ago

1.3.375

9 months ago

1.3.373

10 months ago

1.3.372

10 months ago

1.3.370

10 months ago

1.3.368

10 months ago

1.3.366

10 months ago

1.3.365

10 months ago

1.3.363

10 months ago

1.3.362

10 months ago

1.3.360

10 months ago

1.3.359

10 months ago

1.3.358

10 months ago

1.3.355

10 months ago

1.3.348

10 months ago

1.3.347

10 months ago

1.3.345

11 months ago

1.3.342

11 months ago

1.3.341

11 months ago

1.3.340

11 months ago

1.3.338

12 months ago

1.3.336

12 months ago

1.3.335

12 months ago

1.3.333

12 months ago

1.3.332

12 months ago

1.3.331

12 months ago

1.3.330

12 months ago

1.3.328

1 year ago

1.3.327

1 year ago

1.3.325

1 year ago

1.3.322

1 year ago

1.3.321

1 year ago

1.3.320

1 year ago

1.3.318

1 year ago

1.3.316-1

1 year ago

1.3.316

1 year ago

1.3.314

1 year ago

1.3.313

1 year ago

1.3.311

1 year ago

1.3.310

1 year ago

1.3.309

1 year ago

1.3.308

1 year ago

1.3.306

1 year ago

1.3.304

1 year ago

1.3.302

1 year ago

1.3.300

1 year ago

1.3.297-test-4

1 year ago

1.3.297-test-correct-1

1 year ago

1.3.297-test

1 year ago

1.3.299

1 year ago

1.3.297

1 year ago

1.3.294

1 year ago

1.3.293

1 year ago

1.3.292

1 year ago

1.3.291

1 year ago

1.3.290

1 year ago

1.3.289

1 year ago

1.3.287

1 year ago

1.3.286

1 year ago

1.3.283

1 year ago

1.3.281

1 year ago

1.3.278

1 year ago

1.3.277

1 year ago

1.3.275

1 year ago

1.3.273

1 year ago

1.3.271

1 year ago

1.3.269

1 year ago

1.3.267

1 year ago

1.3.264

1 year ago

1.3.262

1 year ago

1.3.261

1 year ago

1.3.259

1 year ago

1.3.257

1 year ago

1.3.255

1 year ago

1.3.254

1 year ago

1.3.253

1 year ago

1.3.251

1 year ago

1.3.250

1 year ago

1.3.248

1 year ago

1.3.246

1 year ago

1.3.245

1 year ago

1.3.242

1 year ago

1.3.241

1 year ago

1.3.240

1 year ago

1.3.239

1 year ago

1.3.238

1 year ago

1.3.237

1 year ago

1.3.236

1 year ago

1.3.235

1 year ago

1.3.234

1 year ago

1.3.231

1 year ago

1.3.229

1 year ago

1.3.227

1 year ago

1.3.225

1 year ago

1.3.223

1 year ago

1.3.222

1 year ago

1.3.220

1 year ago

1.3.217

1 year ago

1.3.216

1 year ago

1.3.214

1 year ago

1.3.212

1 year ago

1.3.211

1 year ago

1.3.210

1 year ago

1.3.207

1 year ago

1.3.205

1 year ago

1.3.203

1 year ago

1.3.202

1 year ago

1.3.200

1 year ago

1.3.199

1 year ago

1.3.198

1 year ago

1.3.196

1 year ago

1.3.194

1 year ago

1.3.192

1 year ago

1.3.189

1 year ago

1.3.188

1 year ago

1.3.186

1 year ago

1.3.185

1 year ago

1.3.184

1 year ago

1.3.183

1 year ago

1.3.181

1 year ago

1.3.179

1 year ago

1.3.178

1 year ago

1.3.175

1 year ago

1.3.174

1 year ago

1.3.172

1 year ago

1.3.171

1 year ago

1.3.167

1 year ago

1.3.165

1 year ago

1.3.163

1 year ago

1.3.160

1 year ago

1.3.158

1 year ago

1.3.156

1 year ago

1.3.154

1 year ago

1.3.152

1 year ago

1.3.151

1 year ago

1.3.150

1 year ago

1.3.149

1 year ago

1.3.147

1 year ago

1.3.142

2 years ago

1.3.141

2 years ago

1.3.139

2 years ago

1.3.137

2 years ago

1.3.136

2 years ago

1.3.133

2 years ago

1.3.130

2 years ago

1.3.129

2 years ago

1.3.127

2 years ago

1.3.124

2 years ago

1.3.122

2 years ago

1.3.120

2 years ago

1.3.118

2 years ago

1.3.117

2 years ago

1.3.113

2 years ago

1.3.111

2 years ago

1.3.109

2 years ago

1.3.108

2 years ago

1.3.107

2 years ago

1.3.102

2 years ago

1.3.100

2 years ago

1.3.99

2 years ago

1.3.98

2 years ago

1.3.96

2 years ago

1.3.94

2 years ago

1.3.92

2 years ago

1.3.90

2 years ago

1.3.87

2 years ago

1.3.85

2 years ago

1.3.84

2 years ago

1.3.82

2 years ago

1.3.79

2 years ago

1.3.77

2 years ago

1.3.75

2 years ago

1.3.73

2 years ago

1.3.70

2 years ago

1.3.68

2 years ago

1.3.64

2 years ago

1.3.61

2 years ago

1.3.59

2 years ago

1.3.57

2 years ago

1.3.56

2 years ago

1.3.55

2 years ago

1.3.52

2 years ago

1.3.50

2 years ago

1.3.49

2 years ago

1.3.47

2 years ago

1.3.44

2 years ago

1.3.42

2 years ago

1.3.41

2 years ago

1.3.40

2 years ago

1.3.38

2 years ago

1.3.33

2 years ago

1.3.32

2 years ago

1.3.28

2 years ago

1.3.26

2 years ago

1.3.24

2 years ago

1.3.23

2 years ago

1.3.17

2 years ago

1.3.14

2 years ago

1.3.12

2 years ago

1.3.11

2 years ago

1.3.10

2 years ago

1.3.9

2 years ago

1.3.8

2 years ago

1.3.5

2 years ago

1.3.4

2 years ago

1.3.3

2 years ago

1.3.2

2 years ago

1.3.1

2 years ago

1.2.23

2 years ago

1.2.22

2 years ago

1.2.21

2 years ago

1.2.20

2 years ago

1.2.19

2 years ago

1.2.18

2 years ago

1.2.16

2 years ago

1.2.12

2 years ago

1.2.11

2 years ago

1.2.10

2 years ago

1.2.8

2 years ago

1.2.7

2 years ago

1.2.6

2 years ago

1.2.4

2 years ago

1.2.3

2 years ago

1.2.2

2 years ago

1.2.1

2 years ago

1.2.0

2 years ago

1.1.74

2 years ago

1.1.72

2 years ago

1.1.69

2 years ago

1.1.68

2 years ago

1.1.65

2 years ago

1.1.64

2 years ago

1.1.63

2 years ago

1.1.62

2 years ago

1.1.59

2 years ago

1.1.57

2 years ago

1.1.56

2 years ago

1.1.55

2 years ago

1.1.54

2 years ago

1.1.53

2 years ago

1.1.52

2 years ago

1.1.51

2 years ago

1.1.50

2 years ago

1.1.47

2 years ago

1.1.45

2 years ago

1.1.44

2 years ago

1.1.43

2 years ago

1.1.42

2 years ago

1.1.41

2 years ago

1.1.40

2 years ago

1.1.39

2 years ago

1.1.38

2 years ago

1.1.37

2 years ago

1.1.36

2 years ago

1.1.34

2 years ago

1.1.31

2 years ago

1.1.21

2 years ago

1.1.20

2 years ago

1.1.17

3 years ago

1.1.15

3 years ago

1.1.14

3 years ago

1.1.12

3 years ago

1.1.9

3 years ago

1.1.6

3 years ago

1.1.5

3 years ago

1.1.4

3 years ago

1.1.3

3 years ago

1.1.1

3 years ago

1.0.15

3 years ago

1.0.14

3 years ago

1.0.13

3 years ago

1.0.12

3 years ago

1.0.11

3 years ago

1.0.10

3 years ago

1.0.7

3 years ago

1.0.5

3 years ago

1.0.4

3 years ago

1.0.3

3 years ago

1.0.2

3 years ago

1.0.1

3 years ago

1.0.0

3 years ago

0.0.18

3 years ago

0.0.17

3 years ago

0.0.16

3 years ago

0.0.15

3 years ago

0.0.14

3 years ago

0.0.13

3 years ago

0.0.12

3 years ago

0.0.11

3 years ago

0.0.10

3 years ago

0.0.9

3 years ago

0.0.8

3 years ago

0.0.7

3 years ago

0.0.6

3 years ago

0.0.5

3 years ago

0.0.4

3 years ago

0.0.3

3 years ago

0.0.2

3 years ago

0.0.1

3 years ago