async-qps-throttle v1.0.1
async-qps-throttle
This package provides throttling of promise-based work, based on one or both of the amount of concurrent active work and QPS (queries per second). ES6 or higher is required.
Use
Creation
Simply import AsyncThrottle
from this package:
import {AsyncThrottle} from 'async-qps-throttle';
const throttle = new AsyncThrottle({maxOutstanding: 10, maxQps: 10});
The constructor takes an options object hash. The available options are:
maxOutstanding
: The maximum number of work items that may be actively running at any given timemaxQps
: The maximum number of work items that will be started in any given second using a rolling window (note that work may continue to run and won't count against QPS)
The example above creates a throttle that will ensure that no more than 10 work items are ever concurrently running, and no more than 10 work items are executed in any given period of one second.
Providing Work
Work is provided in the form of a function which takes no parameters and returns a promise (the implementation
uses the native ES6 Promise
, see below for more information). The use of a generator function allows the
throttle to determine the appropriate time to actually start the work. Work is provided to the single method
callThrottled
, like so:
...
// Pre-defined function.
function postToSite() {
return xhrPromise.send({...});
}
throttle.callThrottled(postToSite);
// Arrow function.
throttle.callThrottled(() => xhrPromise.send({...}));
...
Where's My Promise?
callThrottled
returns a promise that is immediately available and will eventually settle in the same way as the
promise generated by the work function. If the work function throws, this promise will also be rejected with that error.
...
// Unthrottled call.
const unthrottledPromise = xhrPromise.send({...});
// Throttled call.
const throttledPromise = throttle.callThrottled(() => xhrPromise.send({...}));
// Inline then-ing with throttle.
throttle
.callThrottled(() => xhrPromise.send({...}))
.then(response => {...});
...
When Is Everything Done?
AsyncThrottle
provides two additional methods for figuring out when all work is complete (aside from just keeping
and waiting on the returned promises). The first is whenDrained
, a method that provides a promise that will resolve
at the next moment when all tracked work is complete. This promise does not provide any results, but simply acts as
a way to ensure work has completed. This promise is never rejected.
...
throttle.callThrottled(() => xhrPromise.send({...}));
throttle.callThrottled(() => xhrPromise.send({...}));
throttle.callThrottled(() => xhrPromise.send({...}));
throttle.callThrottled(() => xhrPromise.send({...}));
throttle
.whenDrained()
.then(() => {...});
...
The second is a static method called callAllThrottled
, which works similarly to Promise.all
but will throttle the
array of work items according to throttling options provided with the call.
...
const workFn1 = () => return Promise.resolve(1);
const workFn2 = () => return Promise.resolve(2);
const workFn3 = () => return Promise.resolve(3);
AsyncThrottle
.callAllThrottled([workFn1, workFn2, workFn3], {maxQps: 1})
.then(result => {... result will be [1, 2, 3] ...});
...
The promise returned by callAllThrottled
will be resolved with an array of results that correspond to the input work,
or will be rejected with the first error that any executed work returned.
Details
Outstanding Work Measurement
Outstanding work is measured based on how many concurrent work items are currently running. As an example, suppose there
are work items that takes varying lengths of time to run and the throttle is set up with {maxOutstanding: 2}
. Execution
might proceed as follows:
. = throttled
* = running
! = complete
0ms 1000ms 2000ms 3000ms 4000ms 5000ms 6000ms
+---------+--------+---------+---------+---------+---------+
W1 ********************************************!
W2 **********!
W3 ..........******************************!
W4 ........................................*********!
W5 .............................................********!
QPS Measurement
QPS is strictly measured by the invocation of work, never how long the work takes to complete. As an example, suppose
there are work items that takes 3000ms to run and the throttle is set up with {maxQps: 1}
. Execution would proceed as
follows:
. = throttled
* = running
! = complete
0ms 1000ms 2000ms 3000ms 4000ms 5000ms 6000ms
+---------+--------+---------+---------+---------+---------+
W1 *****************************!
W2 ..........*****************************!
W3 ...................******************************!
W4 .............................******************************!
Note that during execution, at times there are up to 3 concurrent work items.
Promises
This package consumes and returns native ES6 promises (Promise
). If your project uses native promises, read no further.
However, if your project uses an external promise library (e.g., Bluebird), read on.
If running in a strongly-typed environment (e.g., TypeScript), simply wrap promises on the way in and/or the way out:
...
function fancyPromiseGenerator() {return fancyPromise();}
function fancyPromiseConsumer(fancyPromise) {...}
const rawPromise = throttle.callThrottled(() => Promise.resolve(fancyPromiseGenerator()));
fancyPromiseConsumer(FancyPromise.resolve(rawPromise));
On the other hand, if you are running in a weakly-typed environment (e.g., ES6), it may not be necessary to wrap the
promises at all. Just note, the promises returned by this module will be fairly basic (no tap
, finally
, etc.).