smart-downloader v1.2.1
Smart Downloader
Description
HTTP(s) files downloader that can resume your downloads and supports download speed limits (throttle). It also supports file download progress callback to receive progress value in percentage.
Script ensures (creates if missing) that destination dir exists.
You can also extract downloaded archive file, just specify where to. It supports tarball (gzipped as well) and zip files.
If you already know md5 checksum value for the file to be downloaded, you can also validate downloaded file against it.
*Important - it currently uses wget to make file downloads resumable and throttled, so it might not work on all operating systems.
There is a plan to add a fall back to just normal download for unsupported operating systems.
Install
npm install --save smart-downloader
Simple usage
const SmartDownloader = require('smart-downloader');
const downloader = new SmartDownloader();
downloader.download({
uri: 'https://github.com/gegis/smart-downloader/raw/master/test/fixtures/code-1.jpg',
destinationDir: './downloads/'
}, (err, data) => {
if (err) {
console.log(err);
}
console.log(data);
});Advanced usage
A file download with an option to resume and limit download speed to 125 kBps (125 KiloBytes = 1 Megabit) and a custom destination file name:
const SmartDownloader = require('smart-downloader');
const downloader = new SmartDownloader();
downloader.download({
uri: 'https://github.com/gegis/smart-downloader/raw/master/test/fixtures/code-1.jpg',
destinationDir: './downloads/',
destinationFileName: 'advanced-1.jpg',
resumeDownload: true,
downloadSpeedLimit: 125 //value in KiloBytes per second
}, (err, data) => {
if (err) {
console.log(err);
}
console.log(data);
});You can also specify defaults in constructor and specify md5 checksum value for the downloaded file to be verified:
const SmartDownloader = require('smart-downloader');
const downloader = new SmartDownloader({
resumeDownload: true,
downloadSpeedLimit: 25 //value in KiloBytes per second
});
downloader.download({
uri: 'https://github.com/gegis/smart-downloader/raw/master/test/fixtures/code-1.jpg',
destinationDir: './downloads/',
destinationFileName: 'advanced-2.jpg',
md5: 'd4a63031f57bdcafb86ca02100fdd6d2'
}, (err, data) => {
if (err) {
console.log(err);
}
console.log(data);
});You can pass progress callback to get download progress updates:
const SmartDownloader = require('smart-downloader');
const downloader = new SmartDownloader({
resumeDownload: true,
downloadSpeedLimit: 250, //value in KiloBytes per second
progressUpdateInterval: 2000 // value in milliseconds
});
downloader.download({
uri: 'https://unsplash.com/photos/cvBBO4PzWPg/download?force=true',
destinationDir: './downloads/',
destinationFileName: 'advanced-3.jpg',
md5: 'd94f347a514c051cff5a28814ddacb73'
}, (err, data) => {
if (err) {
console.log(err);
}
console.log(data);
}, (err, data) => {
if (err) {
console.log(err);
}
console.log(`Progress: ${data.progress}%`);
});You can pass extract dir to extract (unzip) archived file contents:
const SmartDownloader = require('smart-downloader');
const downloader = new SmartDownloader();
downloader.download({
uri: 'https://github.com/gegis/smart-downloader/raw/master/test/fixtures/code.tar.gz',
destinationDir: './downloads/',
extractDir: './downloads/code'
}, (err, data) => {
if (err) {
console.log(err);
}
console.log(data);
});Specify request Headers (it accepts the array of header strings):
const SmartDownloader = require('smart-downloader');
const downloader = new SmartDownloader();
downloader.download({
uri: 'https://github.com/gegis/smart-downloader/raw/master/test/fixtures/code.tar.gz',
destinationDir: './downloads/',
extractDir: './downloads/code',
headers: ['Accept-Language: "en-us"', "Accept-Encoding: 'gzip, deflate'"]
}, (err, data) => {
if (err) {
console.log(err);
}
console.log(data);
});Specify other wget related options (it accepts the array of options):
const SmartDownloader = require('smart-downloader');
const downloader = new SmartDownloader();
downloader.download({
uri: 'https://github.com/gegis/smart-downloader/raw/master/test/fixtures/code.tar.gz',
destinationDir: './downloads/',
extractDir: './downloads/code',
wgetOptions: ['--no-dns-cache', '--wait=1']
}, (err, data) => {
if (err) {
console.log(err);
}
console.log(data);
});Example of how to stop ("pause") current download process:
const SmartDownloader = require('smart-downloader');
const downloader = new SmartDownloader();
// It returns child process instance
const cmd = downloader.download({
uri: 'https://unsplash.com/photos/cvBBO4PzWPg/download?force=true',
destinationDir: './downloads/',
destinationFileName: 'test-stop.jpg,
downloadSpeedLimit: 300
}, (err, data) => {
if (err) {
// Handling the known error
if (err.message === 'Download error. Signal: SIGTERM') {
console.log('Handled stop);
} else {
console.log(err);
}
}
console.log(data);
});
// After 3 seconds we decide we want to stop it from downloading
setTimeout(() => {
// You can kill it with custom signal, i.e. "SIGINT"
// By default it's SIGTERM signal
// It will cause an error in your callback, make sure to handle it
cmd.kill();
}, 3000);Options
Can be passed either to constructor or download function:
uri- url to download file fromdestinationDir- destination directory, absolute or relative pathresumeDownload- resume download, default:truedownloadSpeedLimit- download speed limit integer value, default:nulldownloadSpeedLimitUnit- download speed limit unit value, default'k'(KiloBytes)progressUpdateInterval- it tells how often to update with download progress, value in ms, default1000destinationFileName- downloaded file name in destination dir, otherwise it will use original file namemd5- if md5 checksum specified, it will verify downloaded file md5 checksum against itextractDir- if specified, it will extract downloaded archive to the specified dirheaders- (array) if specified, it will append headers to requestwgetOptions- (array) if specified, it will append all options related to wget command