sonic-async v0.5.11

Sonic-Async

Sonic-Async is an async flow control javascript library with excellent performance characteristics.
It consistently comes out on top on benchmarking runs. For the API calls that are currently available, they are compatible with Async except waterfall.

Sonic-Async can be installed as a node module with npm.

$npm install sonic-async

You can also use it in a browser running as a very lightweight JS library.

<script type="text/javascript" src="https://npmcdn.com/sonic-async/sonicAsync.min.js"></script>

API Sumary

Listed below are currently available APIs with a brief description of what it is for each.
Click the name of each API for details.

Control Flow

• all - runs all the tasks in parallel regardless what the return staus of each task.
• parallel - runs all the tasks in parallel.
• parallelLimit - runs tasks in parallel with limited concurrency.
• race - runs all the tasks in parallel and the optional done callback returns the result from the 1st completed task.
• series - runs tasks sequencially.
• waterfall - runs tasks sequencially and passes the result from one to the next.
• fastWaterfall - runs tasks sequencially and passes the result from one to the next.

Collections

• map - maps the values in a collection in parallel and returns a new array that contains the mapped values.
• mapAll - maps the values in a collection in parallel and returns a new array that contains the mapped values regardless the mapper's return status.
• mapParallel - same as map.
• mapLimit - maps the values in a collection in parallel with limited concurrency and returns a new array that contains the mapped values.
• mapSeries - maps the values in a collection sequencially and returns a new array that contains the mapped values.
• filter - filters the values in a collection in parallel and returns a new array that contains the filtered values.
• filterParallel - same as filter.
• filterLimit - filters the values in a collection in parallel with limited concurrency and returns a new array that contains the filtered values.
• filterSeries - filters the values in a collection sequencially and returns a new array that contains the filtered values.

Options

This library lets you decide how to handle the multiple task callback invocations. By default, it will print a message to the console.

• E.g.,

  callback called already; err= null ; data= 2

Here, the err and data are the values from 2 arguments of the task callback.

• sonicAsync.silent - if set to false, a message similar to the example above is printed to the console. The default value is false.
• sonicAsync.trace - if set to true, the stack trace is printed to the console and no error is thrown. The default value is false.

• sonicAsync.errout - if set to true and sonicAsync.trace is false, an error is thrown. The default value is false.

• Usage:

  sonicAsync = require('sonic-async');  
  sonicAsync.silent = true;  // no message will be printed

API Details

Control Flow

all(tasks, done_callback)

Runs all the tasks in parallel as if all of them run at the same time. If any one of the tasks passes an error to its callback, the error is stored in the results array instead of calling done_callback immediately. All tasks are run until completion regardless what status they return.

Arguments

• tasks - an array of functions to be executed. Each function is passed a callback(err, result) which it must call on completion with an error err (which can be null) and an optional result value.
• done_callback(err, results) - an optional callback to be executed once all the functions have completed.
  The results argument is an array that contains the values produced by all the tasks with the order aligned with tasks. err is always set to null.

Example

sonicAsync.all([
    function(callback){
        setTimeout(function(){
            callback('error', null);
        }, 2);
    },
    function(callback){
        setTimeout(function(){
            callback(null, 3);
        }, 3);
    }
],
// optional done callback
function(err, results){
    console.log(results); //  ['error', 3] 
});

parallel(tasks, done_callback)

Runs all the tasks in parallel as if all of them run at the same time. If any one of the tasks passes an error to its callback, the done_callback is immediately called with the value of the error. Once all the tasks are done, the results are passed to the done_callback as an array in the same order of tasks.

Arguments

• tasks - an array of functions to be executed. Each function is passed a callback(err, result) which it must call on completion with an error err (which can be null) and an optional result value.
• done_callback(err, results) - an optional callback to be executed once all the functions have completed successfully or an error was return by one of the tasks. The results argument is an array that contains the values produced by all the tasks with the order aligned with tasks.

Example

sonicAsync.parallel([
    function(callback){
        setTimeout(function(){
            callback(null, 2);
        }, 2);
    },
    function(callback){
        setTimeout(function(){
            callback(null, 3);
        }, 3);
    }
],
// optional done callback
function(err, results){
    console.log(results); //  [2, 3] 
});

parallelLimit(tasks, limit, done_callback)

The number of tasks that runs in parallel are capped at limit. A task will be spawn up right away once a task is completed one after another until all the tasks are spawn up. If any one of the tasks passes an error to its callback, the done_callback is immediately called with the value of the error. Once all the tasks are done, the results are passed to the done_callback as an array in the same order of tasks.

Arguments

• tasks - an array of functions to be run. Each function is passed a callback(err, result) which it must call on completion with an error err (which can be null) and an optional result value.
• limit - the concurrency of running tasks is capped at limit.
• done_callback(err, results) - an optional callback to be run once all the functions have completed successfully or when an error was return by any one of the functions. The results argument is an array that contains the values produced by all the tasks with the order aligned with tasks.

Example

sonicAsync.parallelLimit([
    function(callback){
        setTimeout(function(){
            callback(null, 1);
        }, 2);
    },
    function(callback){
        setTimeout(function(){
            callback(null, 2);
        }, 9);
    },
    function(callback){
        setTimeout(function(){
            callback(null, 3);
        }, 12);
    },
    function(callback){
        setTimeout(function(){
            callback(null, 4);
        }, 20);
    },
    function(callback){
        setTimeout(function(){
            callback(null, 5);
        }, 5);
    }
],
//concurrency limit
2,
// optional done callback
function(err, results){
    console.log(results); //  [1, 2, 3, 4, 5] 
});

race(tasks, done_callback)

Runs all the tasks in parallel as if all of them run at the same time. the done_callback is immediately called with the value of either failure or success from the 1st completed task.

Arguments

• tasks - an array of functions to be executed. Each function is passed a callback(err, result) which it must call on completion with an error err (which can be null) and an optional result value.
• done_callback(err, result) - an optional callback to be invoked by the 1st completed task callback.

Example

sonicAsync.race([
    function(callback){
        setTimeout(function(){
            callback(null, 2);
        }, 2);
    },
    function(callback){
        setTimeout(function(){
            callback(null, 3);
        }, 3);
    }
],
// optional done callback
function(err, result){
    console.log(result); //  2 
});

series(tasks, done_callback)

Runs the tasks sequentially, one after another until all the tasks have been completed. If any one of the tasks in the series passes an error to its callback, done_callback is immediately called with the value of the error. Otherwise, done_callback receives the results as an array when all the tasks have been completed.

Arguments

• tasks - an array containing functions to be executed, each function is passed a callback(err, result) it must call on completion with an error err (which can be null) and an optional result value.
• done_callback(err, results) - an optional callback to run once all the functions have been completed successfully or when an error was return by any one of the functions. The results argument is an array that contains the values produced by all the tasks with the order aligned with tasks.

Example

sonicAsync.series([
    function(callback){
        // do something ...
        callback(null, 1);
    },
    function(callback){
        // do some more things...
        callback(null, 2);
    }
],
// optional done callback
function(err, results){
    console.log(results); // [1, 2]
});

waterfall(tasks, done_callback)

Runs the tasks sequentially, each passing its result to the next. However, if any one of the tasks passes an error to its own callback, the next task is not executed, and the done_callback is immediately called with the error.

Arguments

• tasks - an array of functions to be executed, each function is passed a callback(err, result) it must call on completion. The first argument is an error (which can be null) and the 2nd argument, resutl, is passed on to the next task.
• done_callback(err, result) - an optional callback to be run once all the functions have completed. The result from the last task will be passed to done_callback as the 2nd and last argument.

Example

soniAsync.waterfall([
    // the wrapping function always has the signature of function(data, callback)
    function(data, callback) {
        callback(null, 1);
    },
    function(data, callback) {
      // data now equals 1
        callback(null, [2,5]);
    },
    function(data, callback) {
      // data now equals [2,5]
        callback(null, 3);
    }
], function (err, result) {  // the done_callback
    console.log(result ) //  3
});

fastWaterfall(tasks, done_callback)

This API is basically the same as waterfall with the exception that it is not callback safe.
If the task callback is invoked more than once, it may cause one or more of the tasks to be skipped.
Since it does not guard against multiple task callbacks, it runs faster than the regular waterfall API.

map(data, mapfunc, done_callback)

Produces a new array by mapping each value in the data array through the mapfunc function. The mapfunc is called with an item from data and a callback for when it has finished processing. Each of these callbacks takes 2 arguments: an error, and the transformed item from data. If mapfunc passes an error to its callback, the done_callback is immediately called with the error. Since this API applies the mapfunc to each item in parallel, the mapfunc functions can complete in any order. However, the results array will be in the same order as the original data array.

Arguments

• data - an array to iterate over.
• mapfunc(item, callback) - a function to apply to each item in data. The mapfunc is passed a callback(err, transformed_item) which must be called once it has completed with an error (which can be null) and a transformed item.
• done_callback(err, results) - an optional callback which is called when all mapfunc functions have finished, or an error occurs. Results are an array of the transformed items from the data array.

Example

sonicAsync.map([1,2,3], 
    function mapfunc(val, cb) {
        setTimeout(function() {
                cb(null, val*2);
            }, 5
        );
    }, 
    function(err, results){
        console.log(results); //  [2,4,6]
});

mapAll(data, mapfunc, done_callback)

Produces a new array by mapping each value in the data array through the mapfunc function. The mapfunc is called with an item from data and a callback for when it has finished processing. Each of these callbacks takes 2 arguments: an error, and the transformed item from data. Since this API applies the mapfunc to each item in parallel, the mapfunc functions can complete in any order. However, the results array will be in the same order as the original data array. The done_callback is called when all the values in data has been mapped regarless the return status of manpfunc.

Arguments

• data - an array to iterate over.
• mapfunc(item, callback) - a function to apply to each item in data. The mapfunc is passed a callback(err, transformed_item) which must be called once it has completed with an error (which can be null) and a transformed item.
• done_callback(err, results) - an optional callback which is called when all mapfunc functions have finished. Results are an array of the transformed items from the data array.

Example

sonicAsync.map([1,2,3], 
    function mapfunc(val, cb) {
        setTimeout(function() {
            if( val === 2 )
                cb('error', null);
            else
                cb(null, val*2);
            }, 5
        );
    }, 
    function(err, results){
        console.log(results); //  [2,'error',6]
});

mapLimit(data, mapfunc, limit, done_callback)

This is the same as map except that the concurrency is capped at limit. When a mapfunc is done mapping an item, another mapfunc is spawn up to map the next item in the data array.

Arguments

• data - an array to iterate over.
• limit - the concurrency of running mapfuncs is capped at limit.
• mapfunc(item, callback) - a function to apply to each item in data. The mapfunc is passed a callback(err, transformed_item) which must be called once it has completed with an error (which can be null) and a transformed item.
• done_callback(err, results) - an optional callback which is called when all mapfunc functions have finished, or an error occurs. Results is an array of the transformed items from the data array.

Example

sonicAsync.map([1,2,3], 
    // maximum mapfuncs running is 2
    2,
    function mapfunc(val, cb) {
        setTimeout(function() {
                cb(null, val*2);
            }, 5
        );
    }, 
    function(err, results){
        console.log(results); //  [2,4,6]
});

mapParallel(data, mapfunc, done_callback)

This is exactly the same as map.

mapSeries(data, mapfunc, done_callback)

This is a special case of mapLimit with limit set to 1.

filter(data, filterfunc, done_callback)

Produces a new array by filtering each value in the data array through the filterfunc function. The filterfunc is called with an item from data and a callback for when it has finished processing. Each of these callbacks takes 2 arguments: an error, and a boolean. If filterfunc passes an error to its callback, the done_callback is immediately called with the error. Since this API applies the filterfunc to each item in parallel, the filterfunc functions can complete in any order. However, the order of the items in the results array is aligned with the original data array.
The results array contains the items if any where the filterfunc callback's 2nd argument is set to true.

Arguments

• data - an array to iterate over.
• filterfunc(item, callback) - a function to apply to each item in data. The filterfunc is passed a callback(err, boolean) which must be called once it has completed with an error (which can be null) and a true or false value.
• done_callback(err, results) - an optional callback which is called when all filterfunc functions have finished, or an error occurs. Results is an array of the filtered items from the data array.

Example

sonicAsync.filter([1,2,3], 
    function filterfunc(val, cb) {
        setTimeout(function() {
                cb(null, val%2);
            }, 5
        );
    }, 
    function(err, results){
        console.log(results); //  [1,3]
});

filterLimit(data, filterfunc, limit, done_callback)

This API is the same as filter except that the concurrency is capped at limit. When a filterfunc is done filtering an item, another filterfunc is spawn up to filter the next item in the data array.

Arguments

• data - an array to iterate over.
• filterfunc(item, callback) - a function to apply to each item in data. The filterfunc is passed a callback(err, boolean) which must be called once it has completed with an error (which can be null) and a true or false value.
• limit - the concurrency of running filterfuncs is capped at limit.
• done_callback(err, results) - an optional callback which is called when all filterfunc functions have finished, or an error occurs. Results is an array of the filtered items from the data array.

Example

sonicAsync.filter([1,2,3], 
    2,
    function filterfunc(val, cb) {
        setTimeout(function() {
                cb(null, val%2);
            }, 5
        );
    }, 
    function(err, results){
        console.log(results); //  [1,3]
});

filterParallel(data, filterfunc, done_callback)

This is exactly the same as filter.

filterSeries(data, filterfunc, done_callback)

This is a special case of filterLimit with limit set to 1.

Benchmarks

The code for benchmarking is a copy from bluebird's benchmark. A correction has been made to remove the bias against async style libraries in parallel bechmarking. See bluebird issue #985 for details.

DoxBee sequential

Run: npm run benchw

results for 10000 parallel executions, 1 ms per I/O op

Parallel

Run: npm run benchp

results for 10000 parallel executions, 1 ms per I/O op

Concurrency

Run: npm run benchl
Total number of tasks: 25
Concurrency: 5

results for 10000 parallel executions, 1 ms per I/O op