@thy3634/chunked-uploader v0.1.0

chunked-uploader

Split file into chunks and upload. resumable, offline aware. Works on node, browser, and workers.

Demo

A Master-Worker pattern to upload a file in chunks. A master worker to process file, split it into chunks and upload, Using several workers to calculate MD5 of chunk.

Include both backend and frontend.

Or git clone this repository and run pnpm dev to see the demo.

Usage

Install package:

# ✨ Auto-detect
npx nypm install @thy3634/chunked-uploader

# npm
npm install @thy3634/chunked-uploader

# yarn
yarn add @thy3634/chunked-uploader

# pnpm
pnpm install @thy3634/chunked-uploader

# bun
bun install @thy3634/chunked-uploader

Import:

ESM (Node.js, Bun)

import { ChunkedUploader } from "@thy3634/chunked-uploader";

CommonJS (Legacy Node.js)

const { ChunkedUploader } = require("@thy3634/chunked-uploader");

conditional exports

CDN (Deno, Bun and Browsers)

import { ChunkedUploader } from "https://esm.sh/@thy3634/chunked-uploader";

1. Create a new instance of ChunkedUploader:

// usually you need to get a upload ID from server
const { id } = await ofetch('/api/chunked-upload', {
    method: 'POST',
    body: {
        fileSize: file.size,
        filename: file.name
    }
})
// create a new instance of ChunkedUploader
const uploader = new ChunkedUploader(file, async ({ buffer, index, start, end, digest }) => ofetch(`/chunked-upload/${id}/${index}`, {
                method: 'PUT',
                body: await buffer,
                headers: {
                    'Range': `bytes=${start}-${end - 1}`,
                    'Content-Digest': `md5=:${hexStringToBase64(await digest)}:`,
                },
                retry: 1,
            }))

2. Listen to events:

uploader.addEventListener('progress', ({ loaded, total }) => {
    console.log(`progress: ${loaded}/${total}`)
})
uploader.addEventListener('error', (event) => {
    console.error(uploader.error)
})
uploader.addEventListener('success', (event) => {
    console.log('success')
})

3. Start uploading:

uploader.start()
// or await for completion, which will return the response list, or throw error
await uploader.start()

API

Class ChunkedUploader

Extends EventTarget

Constructor

Methods and Properties

start(): Promise<unknown[]>

Start the upload, if the upload is already started, do nothing, otherwise:

• property status will be set to 'pending'
• event start will be dispatched
• if onLine is false, pause

Parameters

pause(): boolean

Pause the upload, if it is not uploading, do nothing, otherwise:

• property status will be set to 'paused'
• method abort will be called
• event pause will be dispatched

resume(): Promise<unknown[]> | false

Resume the upload, if it is not paused, do nothing, otherwise:

• property status will be set to 'pending'
• event resume will be dispatched

store(): FileInfo & { chunks: Chunk[] }

Get the file information and chunks so that you can store them and reconstruct the uploader later.

status: 'idle' \| 'pending' \| 'paused' \| 'error' \| 'success'

The current status of the upload process. read-only.

error: Error | undefined

The error that occurred during the upload.

total: number

The total number of chunks. read-only.

loaded: number

The number of chunks that have been uploaded. read-only.

hash: string

A promise that resolves to the MD5 hash (hex) of the file's data. read-only.

chunks: Chunk[]

The chunks array. read-only.

onLine: boolean

Is network online. If navigator is available, use the value of navigator.onLine as the default value, otherwise true.

• When it is set to false, the upload will be paused. When it is set to true, the upload will be resumed.
• If window is available, automatically update, and pause/resume the upload.

abort(): boolean

Abort the upload, if it is not uploading, do nothing, otherwise:

• property status will be set to 'error'
• event 'abort' will be dispatched
• property error will be set to an Error

Events

• start: dispatched when the upload has started.
• progress: dispatched periodically as any chunk uploaded.
• pause: dispatched when the upload pauses.
• resume: dispatched when the upload resumes.
• error: dispatched when an error occurs during the upload.
• success: dispatched when the upload completes successfully.
• end: dispatched when the upload has completed, successfully or not.
• digestprogress: Fired periodically as file digesting

// you can listen to these events using the `on${EventType}` method:
uploader.onpregress = (event) => {
    console.log('progress', event.loaded / event.total)
}
// or using the `addEventListener` method:
uploader.addEventListener('digestprogress', (event) => {
    console.log('digestprogress', event.loaded / event.total)
})

Types

ChunkedUploaderOptions

Http headers suggestions:

• Range
• Content-Digest

Chunk

FileInfo

Hasher

Development

• Clone this repository
• Install latest LTS version of Node.js
• Enable Corepack using corepack enable
• Install dependencies using pnpm install
• Run interactive tests using pnpm dev

License

Published under the MIT license. Made by community 💛

🤖 auto updated with automd