@lemonmade/web-worker v0.0.2-beta.4
@shopify/web-worker
Tools for making web workers fun and type-safe.
Installation
$ yarn add @shopify/web-worker
Usage
This library contains three parts that must be used in tandem:
- A babel plugin provided by
@shopify/web-worker/babel
- A webpack plugin provided by
@shopify/web-worker/webpack
- The public API of the package provided by
@shopify/web-worker
App code
Browser
The main thread will use the createWorker
function provided by this library to create a function that can build "smart" workers. Developers should pass this function an async import()
for the module they wish to make into a worker:
import {createWorker} from '@shopify/web-worker';
const makeWorker = createWorker(() => import('./worker'));
When the resulting function is called, it will return a worker "instance". This instance is directly connected to the worker code, so you are expected to call it as if it were simply that external module. Note that there is one difference: any return values from the worker will be promises, because they are retrieved via message passing (if you use TypeScript, you’ll see the correct return types automatically).
const worker = makeWorker();
// Assuming worker was:
// export function hello(name) {
// return `Hello, ${name}`;
// }
const result = await worker.hello('world'); // 'Hello, world'
Note that more complex workers are allowed; it can export multiple functions, including default exports, and it can accept complex argument types with some restrictions:
const worker = makeWorker();
// Assuming worker was:
// export default function hello(name) {
// return `Hello, ${name}`;
// }
//
// export function getName(user) {
// return user.getDisplayName();
// }
const result = await worker.default(
await worker.getName({
getDisplayName: () => 'Tobi',
}),
); // 'Hello, Tobi'
Worker
Your worker can be written almost indistinguishably from a "normal" module. It can import other modules (including async import()
statements), use modern JavaScript features, and more. The exported functions from your module form its public API, which the main thread code will call into as shown above. Note that only functions can be exported; this library is primarily meant to be used to create an imperative API for offloading work to a worker, for which only function exports are needed.
As noted in the browser section, worker code should be mindful of the limitations of what can be passed into and out of a worker function.
Your worker functions should be careful to note that, if they accept any arguments that include functions, those functions should at least optionally return a promise. This is because, when this argument is passed from the main thread to the worker, it can only pass a function that returns a promise. To help you make sure you are respecting this condition, we provide a SafeWorkerArgument
helper type you can use for all arguments that your worker accepts.
import {SafeWorkerArgument} from '@shopify/web-worker';
export function greet(name: SafeWorkerArgument<string | () => string>) {
// name is `string | (() => string | Promise<string>)` because a worker
// can synchronously pass a `string` argument, but can only provide a
// `() => Promise<string>` function, since it will have to proxy over
// message passing. Note that `() => string` is still allowed because
// it could still be valid for another function in the worker to call
// with a function of that type.
return (
typeof name === 'string'
? `Hello, ${name}`
: Promise.resolve(name()).then((name) => `Hello, ${name}`)
);
}
The same memory management concerns apply to the worker as they do on the main thread.
Limitations
TODO
Memory
TODO
Tooling
To use this library, you must use webpack. When configuring webpack, include the Babel plugin this library provides for any modules that might contain a call to createWorker()
, and include the WebWorkerPlugin
exported by @shopify/web-worker/webpack
:
import {WebWorkerPlugin} from '@shopify/web-worker/webpack';
const webpackConfig = {
// rest of webpack config...
plugins: [new WebWorkerPlugin()],
module: {
rules: [
{
test: /\.js/,
exclude: /node_modules/,
use: [
{
loader: 'babel-loader',
options: {
babelrc: false,
plugins: [require.resolve('@shopify/web-worker/babel')],
},
},
],
},
],
},
};
The WebWorkerPlugin
helps coordinate a few things during webpack compilation and allows you to pass options to the loader, which is otherwise completely hidden. The following options are accepted by the WebWorkerPlugin
constructor:
globalObject
: allows you to customize theoptions.output.globalObject
option passed to the child Webpack compiler. By default, this is set toself
, which works well in workers, but if you have a different global object that must be used, you can pass it here.plugins
: an array of webpack plugins to include in the child compilation of the worker. This library automatically includes a few plugins that will generate output code appropriate for workers.
How does it work?
The @shopify/web-worker/babel
Babel plugin looks for any instances of calling createWorker
. For each one, it looks for the nested import()
call, and then for the imported path (e.g., ./worker
). It then replaces the first argument to createWorker
with an import for the worker module that references a custom Webpack loader:
import {createWorker} from '@shopify/web-worker';
createWorker(() => import('./worker'));
// becomes something like:
import * as workerStuff from '@shopify/web-worker/webpack-loader!./worker';
createWorker(workerStuff);
When webpack attempts to process this import, it sees the loader syntax, and passes the worker script to this package’s custom webpack loader. This loader does most of the heavy lifting. It creates an in-memory module (using information it pulls off the WebWorkerPlugin
instance it finds in the Webpack compiler) that exposes the worker API using the expose()
function from this library:
// This is the imaginary module the loader creates and compiles
import {expose} from '@shopify/web-worker';
import * as api from './worker';
expose(api);
This imaginary module is then compiled using a child compiler in Webpack. The loader then takes the resulting asset metadata from compiling the worker, and makes that information the exported data from the original ./worker
module. Finally, createWorker()
takes this metadata (which includes the main script tag that should be loaded in the worker) and, when called, creates a new Worker
instance using a Blob
that simply importScripts
the main script for the worker.
TODO: how the proxying works