Google-cloud-storage-bulk NPM

google-cloud-storage-bulk

google cloud storage bulk upload

Install

npm install google-cloud-storage-bulk

yarn:

yarn add google-cloud-storage-bulk

Usage

const GCS = require('google-cloud-storage-bulk');

const gcs = new GCS({
  projectId,
  bucketName: 'my-bucket',
  concurrency: 100,
  hashStrategy: 'file',
  retries: 3,
  subdirectory: 'application-name',
  uploadOptions = {}
});

// ... in async function
await gcs.uploadFiles(someDirectory);

Options

projectId - (required) - google cloud project id (see project reference for more details)
bucketName - (required) - cloud storage bucket name
concurrency - (optional - default: 250) - upload concurrency limits
retries - (optional - default: 3) - upload retry attempts
hashStrategy - (required - options: none, file, subdirectory) - described in detail below
subdirectory - (optional) - subdirectory within cloud storage bucket to push content into
uploadOptions - (optional) - extends @google/cloud-storage upload options

`hashStrategy`

`none`

No hashing to file structure, just push content as is. Google cloud does handle metageneration that you can take advantage of for versioning.

Before:
|_directory_to_upload
  |_a.js
  |_b.js
  |_c.css
  
After:
|_subdirectory
  |_a.js
  |_b.js
  |_c.css

`file`

Hash per file currently using sha1 algorithm.

Before:
|_directory_to_upload
  |_a.js
  |_b.js
  |_c.css
  
After:
|_subdirectory
  |_a.a9993e364706816aba3e25717850c26c9cd0d89d.js
  |_b.924f61661a3472da74307a35f2c8d22e07e84a4d.js
  |_c.bcb8c41b803b91661b5e6ee45362f47df368a731.css
  |_asset-manifest.json <-- references initial file to their new hash complement

`subdirectory`

Hash content of the directory being uploaded and use this directory hash as the bucket subdirectory.

Before:
|_directory_to_upload
  |_a.js
  |_b.js
  |_c.css
  
After:
|_subdirectory
  |_924f61661a3472da74307a35f2c8d22e07e84a4d
    |_a.js
    |_b.js
    |_c.css