Worker-parrot-party NPM

Worker Parrot Party!

This library has as purpose the creation of a pool of NodeJs workers (Parrots) using Typescript.

Installation

NPM

npm install --save worker-parrot-party

Motivation

The main purpose of this library is to simplify the usage of Worker Threads with typescript. Additionally, to offer a more flexible way to create worker threads with more complex logic.

How it works

The library dinamically creates a worker script typescript file and compiles this on runtime. Finally we use the Worker Node js library to spawn a thread with the automatically generated file.

NOTE: At the moment, all Parrots in the pool will share the same logic of execution. There is no way to dinamically change the logic

Usage

Import declaration

Javascript

const workerParrotParty = require('worker-parrot-party')

Typescript

import { PoolParty } from 'worker-parrot-party'

Examples

Typescript

import { PoolParty } from 'worker-parrot-party'

// Define an async function task to execute in the threads
async function veryComplexTask (myVar: string): string {
    ...// my complex implementation
}

// Define the pool configuration
const poolPartyConfig = {
    partySize: 2, // number of parrots to spawn
    basePath: __dirname, // current directory
    task: veryComplexTask,
    onSuccess: (result: any) => { console.log('All good in the hood!', result: any) },
    onError: (error: any) => console.error
}

// Create Instance
const poolParty: PoolParty = new PoolParty(poolPartyConfig)

// Spawn the parrot in the pool
poolParty.spawnParrots().then(() => {
    // Run your task in a thread
    poolParty.run('myNiceVariable')
})

// NOTE: You may use async/await

Complex process

Sometimes we need to be able to do a very complex process that may require helper functions.

Scenario. Let's say you need to process thousands of data and need to make a request for all of those results. You can make the definition of all of that process and "inject" the logic to the worker thread, to delegate that logic to an independent process.

import { PoolParty } from 'worker-parrot-party'

// ****************************************************
// This is the isolated logic we want our worker to execute
import { SomeLibrary } from 'SomeLibrary'

async function helperFunction1 () {...}
function helperFunction2() {...}

// Define an async function task to execute in the threads
async function veryComplexTask (myVar: string): string {
    await helperFunction1()
    helperFunction2()
    ...// my complex implementation
}
// ****************************************************

// Define the pool configuration
const poolPartyConfig = {
    partySize: 2, // number of parrots to spawn
    basePath: __dirname, // current directory
    task: veryComplexTask,
    libraryDeclaration: [
        {
            name: 'SomeLibrary',
            importDeclaration: '{ SomeLibrary }'
        }
    ],
    helpers: [helperFunction1, helperFunction2],
    onSuccess: (result: any) => { console.log('All good in the hood!', result: any) },
    onError: (error: any) => console.error
}

// Create Instance
const poolParty: PoolParty = new PoolParty(poolPartyConfig)

// Spawn the parrot in the pool
poolParty.spawnParrots().then(() => {
    // Run your task in a thread
    poolParty.run('myNiceVariable')
})

Pool configuration variables

task REQUIRED: The function to be executed by the worker parrots.
partySize REQUIRED: The number of parrots to be used.
onSuccess REQUIRED: Callback to be executed when all the tasks are completed.
onError REQUIRED: Callback to be executed when an error occurs.
retryInterval OPTIONAL: Interval in milliseconds, for the pool to retry a task. If not specified the default value is 1s.
basePath OPTIONAL: Path to where the automatic generated scripts are stored. Additionally the compiled script is also stored in this path. Default value is ./.
compiledFolderName OPTIONAL: The name of the folder where the compiled scripts are stored. Default value is ./worker-script-dist.
libraryDeclaration OPTIONAL: Specification how a certain library should be imported into the worker script. Default is empty string.
helpers OPTIONAL: Specification of helper function to be used in the worker script.
resourceLimits OPTIONAL: Worker threads resource limits.