mr-downloader v0.0.1

s3-stream-download

Multipart streaming download from S3 with progress watch

Features

Installation

  $ npm i git+https://github.com/medroomdev/s3-stream-download.git --save

Usage

var AWS = require('aws-sdk');
var S3StreamDownload = require('./index');
var fs = require('fs');

var s3 = new AWS.S3({
    apiVersion: '2006-03-01',
    region: '<REGION>',
    signatureVersion: 'v4',
    s3DisableBodySigning: false
});

let download = new s3StreamDownload(s3, { Bucket: '<BUCKET>', Key: '<KEY>' }, { downloadChunkSize:  1024 * 1024, concurrentChunks: 15, 
    onLoad: (totalSize, chunkSize) => {
        // called after totalSize and chunkSize of file is calculated
    }, 
    onPart: () => {
        // called after each part is downloaded
    }
});

download.pipe(window.require('fs').createWriteStream(path.join(app.getAppPath(), '<FILEPATH>')));

download.on('end', () => {
    console.log("finished");
});

Documentation

new S3StreamDownload(s3, s3Params, options)

Creates a new instance of a multipart download stream. This is a readable stream that can be piped into other streams.

Arguments

• s3 - Configured aws-sdk s3 instance. Should be preconfigured with any credentials.
• s3Params - Params object that would normally be passed to s3.getObject()
• options - Options
  • downloadChunkSize - Size of each chunk in bytes. Defaults to 5MB.
  • concurrentChunks - Number of chunks to download concurrently. Defaults to 5.
  • retries - How many times a failed chunk should be retried before failing the entire download. Retries will exponentially backoff to allow for recovery. Defaults to 5.
  • onLoad - Callback for when the information of the object is retrieved from s3.
  • onPart - Callback for when a part is downloaded.

People

Raphael Rosa from MedRoom

based on the package created by Chris Kinsman from PushSpring

License

MIT