Downloadex NPM

downloadex

Download and extract files

See download-cli for the command-line version.

This is a drop-in replacement for the original download, which many projects depend on to download binary files or third-party source code.

The problem for the original version is that it always tries to download the resources, without cache, and not able to use mirrors. This is terrible when used in slow / poor network environments, e.g. in China.

With this replacement, you can easily rewrite the URLs before downloading, by creating an .downloadrc file in your home directory. See Rewriting for more details.

If you are using yarn, simply adding this to your package.json will work:

{
  "resolutions": {
    // one of:
    "download": "https://registry.npmjs.org/downloadex/-/downloadex-8.1.0.tgz", // recommended for universal usage
    "download": "https://registry.npm.taobao.org/downloadex/download/downloadex-8.1.0.tgz",
    "download": "gera2ld/download#v8.1.0", // tag will be fetched every time on install
  }
}

Install

$ npm install downloadex

Usage

const fs = require('fs');
const download = require('downloadex');

(async () => {
	await download('http://unicorn.com/foo.jpg', 'dist');

	fs.writeFileSync('dist/foo.jpg', await download('http://unicorn.com/foo.jpg'));

	download('unicorn.com/foo.jpg').pipe(fs.createWriteStream('dist/foo.jpg'));

	await Promise.all([
		'unicorn.com/foo.jpg',
		'cats.com/dancing.gif'
	].map(url => download(url, 'dist')));
})();

Proxies

To work with proxies, read the got documentation.

Rewriting

This feature allows downloading from mirrors, by supporting URL rewriting.

Create a .downloadrc in you home directory or somewhere else that can be searched by cosmiconfig, with prefix replacing rules. For instance, rewriting the URLs to download image processing binary files:

{
  "rewrite": {
    "https://raw.githubusercontent.com/imagemin/cwebp-bin/": "https://npm.taobao.org/mirrors/cwebp-bin/",
    "https://raw.githubusercontent.com/imagemin/mozjpeg-bin/": "https://npm.taobao.org/mirrors/mozjpeg-bin/",
    "https://raw.githubusercontent.com/imagemin/pngquant-bin/": "https://npm.taobao.org/mirrors/pngquant-bin/"
  }
}

By using a JavaScript config file .downloadrc.js instead of a JSON one, you can also provide rewrite as a function. For example, rewriting GitHub URLs to jsdelivr:

module.exports = {
  rewrite(uri) {
    const matches = uri.match(/^https:\/\/raw\.githubusercontent\.com\/([^/]+\/[^/]+)\/([^/]+)\/(.*)$/);
    if (matches) {
      const [, repo, tag, path] = matches;
      return `https://cdn.jsdelivr.net/gh/${repo}@${tag}/${path}`;
    }
  },
};