Warp NPM | npm.io

Warp

Warp is an API allowing easy parallel execution of javascript code, at the same physical time as the main javascript flow and as other flows.

// runs `f` in parallel
warp.call(f, cb);

Installing

Warp module is an API and is not providing support for execution. You will need to acquire a Warp runtime to use it.

To add Warp API to your project:

$ npm install warp

How to use it

Warp offers two main functions starting a warp task: warp.call and warp.callAsPromise. We say that they perform a warp call. By extension, we also say that code warps a function or that the function is warped.

Here is a very simple illustrative code sample where a CPU intensive function (primeDecomposition) is executed in parallel by a warp call keeping the event loop free for any other processing.

The first example uses a callback-based function, whereas the second uses a Promise-based function.

With callback

const warp = require('warp');

function primeDecomposition(n, done) {
  let f = [];
  for (let i = 2; i <= n; i++)
    while (n % i === 0) {
      f.push(i);
      n /= i;
    }
  done(f);
}

warp.call(primeDecomposition, 9007199254740991,
    (a) => console.log(`Value: ${a}`));

Prints out

Value: 6361,69431,20394401

With promise

const warp = require('warp');

function primeDecomposition(n) {
  let f = [];
  for (let i = 2; i <= n; i++)
    while (n % i === 0) {
      f.push(i);
      n /= i;
    }
  return f;
}

let a = await warp.callAsPromise(primeDecomposition, 9007199254740991);
console.log(`Value: ${a}`);

Prints out

Value: 6361,69431,20394401

How it works

When you warp call a function, the execution context (i.e. everything reachable from the function scope) is captured, then this context and the warped function are sent to a distinct execution flow (parallel flow).

Once the warp task is finished, the parallel context is captured and sent back to the main javascript flow. Then, in an event task of the main flow, the context is updated, and:

If the warp call is call, the callback is executed in the main flow, with the arguments given to the callback in the parallel flow.
If the warp call is callAsPromise, the Promise returned by callAsPromise is resolved or rejected with the value or the exception obtained at the end of the parallel flow.

Scope

The warped functions can read and modify anything reachable from their scope.

In the following sample, the variable o is defined as a free variable (outside of the function). The warped function can read it, and also modify the object. Modifications made to o in the parallel flow are applied and visible in the main flow.

const warp = require('warp');

let o = {};
function primeDecomposition(n) {
  let f = [];
  for (let i = 2; i <= n; i++)
    while (n % i === 0) {
      f.push(i);
      n /= i;
    }
  o.result = f;
}

await warp.callAsPromise(primeDecomposition, 9007199254740991);
console.log(`Value: ${o.result}`);

Prints out

Value: 6361,69431,20394401

Shared resources and concurrency

Warp offers no new synchronization or semaphore mechanism. If you need to manage concurrent access to a value, you'll have to keep the modifying code in the main flow.

In the following example, we compute several prime decompositions in parallel.

const warp = require('warp');

let primeDecompositions = [];

function primeDecomposition(index, n) {
  let f = [];
  for (let i = 2; i <= n; i++)
    while (n % i === 0) {
      f.push(i);
      n /= i;
    }
  primeDecompositions[index] = f;
}

let promises = []
for(let i = 0; i < 8; i++) {
  promises.push(warp.callAsPromise(primeDecomposition, i, 9007174991 + i));
}

await Promise.all(promises);

console.log(`Prime decompositions: %j`, primeDecompositions);

Prints out

Prime decompositions: [17,131,4044533,2,2,2,2,3,23,31,263183,12071,746183,2,37,53,2296577,3,5,7,13,41,227,709,2,2,2251793749,59,109,1400587,2,3,3,3,19,43,204161]

But you need to be carefull about parallel access to shared variables or properties.

For instance, the following example shows bad usage of a shared value causing indetermism. There, you don't know in which order the additions will be made, but it is guaranteed that each compute sees the initial value of v as 4. This means you don't know what the final value of v will be, you just know it will be either 7 or 8.

const warp = require('warp');

let v = 4;
function compute(num) {
  // compute increment from input
  let inc = num;
  v += inc;
  return;
}

let promises = []
promises.push(warp.callAsPromise(compute, 3));
promises.push(warp.callAsPromise(compute, 4));

await Promise.all(promises);

console.log(`Value: ${v}`);

To obtain the correct behavior in this example, you'll need to apply the changes to the shared variable in the main flow by using the .then of the promise.

const warp = require('warp');

let v = 4;
function compute(num) {
  // compute increment from input
  let inc = num;
  return inc;
}

let promises = []
promises.push(warp.callAsPromise(compute, 3).then((inc) => v += inc));
promises.push(warp.callAsPromise(compute, 4).then((inc) => v += inc));

await Promise.all(promises);

console.log(`Value: ${v}`);

Prints out

Value: 11

This way it'll end up with 11 as the final value.

Definitions

Parallel flow

A parallel flow is a javascript execution flow distinct from the main javascript execution flow. The main flow and some parallel flows are running at the same physical time.

Warp task

A warp task is started by a warp call (a call to warp.call or warp.callAsPromise). The task is executed in a parallel flow. It starts by executing the given function and continues until the end of the task is detected.

The end of the task is detected when:

an exception is thrown by the task

If the task was started by warp.callAsPromise:
- the function given to callAsPromise has returned
- And, if its returned value is a Promise, it is fulfilled or rejected.
If the task was started by warp.call
- The callback is called,
- And the function given to call has returned.

Note that this doesn't mean that return was executed, it means that the execution has returned to its caller. For example in the following code the end is detected during the line a = await 3 when await yields and a is still 2.

let a = 0;
async function compute(done) {
  a = 1;
  done();
  a = 2;
  a = await 3; // End is here at `await` before assignation of `a`
  return;
}

Warning: If the task were to attempt to execute code after the end of execution were detected (e.g. with a setImmediate, setTimeout, ...), then the behavior would be unspecified. This means that if the function given to call is starting any asynchronous processing, calling the callback must be the very last thing done by the task.

Execution context

An execution context is referring to everything accessible from the scope of the considered function at a given point of the execution. This includes