@itentialopensource/archive-job-data NPM

Archive Job Data

Archive Job Data

Overview

This pre-built allows IAP users to archive the jobs and tasks collections to keep IAP performant. It also allows the users to restore the archived jobs and tasks. Users have the ability to choose:

The number of days older than which, all qualifying data will be archived.
Custom names for the archive collections to store the matching jobs and tasks documents.
To keep or delete the matching documents from the original jobs and tasks collections after backing them up.
Restore the archived jobs and tasks documents and delete the archived collections.

Users can use the included Operations Manager item to schedule the pre-built to run at regular intervals.

Notes:

For every run, new jobs and tasks collections would be created with the current timestamps appended to the end of the jobs and tasks collection names put in by the user.
Only jobs and tasks related to jobs that are either in complete or canceled state will be considered for archiving. Jobs that are in errored or incomplete state will be left untouched.
Getting an RangeError [ERR_OUT_OF_RANGE] typically indicates that your bundle size is too large. Try setting it lower within Operations Manager automation.
Very large databases with a lot of jobs and tasks to archive will typically result in multiple archive collections within the database.

Installation Prerequisites

Users must satisfy the following prerequisites:

Itential Automation Platform
- ^2023.1
Adapter DB Mongo
- ^0.5.7

How to Install

To install the Workflow Project:

Verify you are running a supported version of the Itential Automation Platform (IAP) as listed above in the Supported IAP Versions section in order to install the Workflow Project.
Import the Workflow Project in Admin Essentials.

How to Run

Use the following steps to run the pre-built:

Once the pre-built is installed as outlined in the How to Install section above, go to Operations Manager and run the Archive Jobs and Tasks Collections automation.
In the Copy/Move Jobs data to Collection and the Copy/Move Tasks data to Collection fields, enter the names for collections you want the matching jobs and tasks collection documents to be copied to.
In the Copy/Move data from before # days field, enter the number of days (from the current date) oolder than which you want data to be archived. Note: Please enter a positive number
In the MongoDB Adapter dropdown menu, select the adapter that is connected the MongoDB that contains the Tasks and Jobs Collections that you want to create an archive of.
In the Batch Size field, select how many jobs you want to archive at a time.
Check the Create Archive Collections checkbox if you want to create an archive collection that will be stored in the MongoDB. Unchecking this box will mean no archive will be created.
Check the Delete copied data from collections checkbox if you want to discard the matching documents from the original jobs and tasks collections after they have been backed up.
Check the Verbose checkbox if you want to see the results of the pre-built at the end of its run and click on START and then on VIEW JOB.

To restore the archived jobs and tasks, run the Restore Archived Jobs and Tasks workflow.

Note: The adapter ID for mongoDB in all the respective tasks of the workflow must be changed accordingly.

Inputs

The inputs for the pre-built are outlined in the following table. To run the workflow directly from IAP/API call (and not through Operations Manager), please encapsulate all of these inputs into a root object with the key formData.

Inputs for Restore Archived Jobs and Tasks workflow are defined below:

{
  "formData": {
    "createArchiveCollections": true,
    "deleteCopiedDataFromCollections": true,
    "verbose": true,
    "copyMoveJobsDataToCollection": "jobs_archive",
    "mongoDbAdapter": "mongoPronghorn",
    "copyMoveDataOlderThanDays": 3,
    "copyMoveTasksDataToCollection": "tasks_archive",
    "batchSize": 2000
  }
}

Sample Input for Workflow `Restore Archived Jobs and Tasks`

{
  "deleteFromArchiveCollections": true,
  "archiveJobsCollectionName": "Archive_Jobs_2023-08-03T21:17:11.079Z",
  "archiveTasksCollectionName": "Archive_Tasks_2023-08-03T21:17:11.079Z",
  "numberOfArchivedCollections": 9
}