@transcend-io/conflux

Conflux

Build and read zip files with whatwg streams in the browser.

• Blazing Fast
• Compatibility
• Examples
• Usage
  • Importing Conflux
  • Creating a ZIP
  • Reading ZIP files
• Supporting Legacy Browsers
• License

Blazing Fast

• ~55 kB import
• Uses streams, minimizing memory overhead

Compatibility

Chrome
Safari
Edge
Firefox

Examples

• Writing zips
• Reading zips

Usage

Importing Conflux

npm install --save @transcend-io/conflux

// Reader parses zip files, Writer builds zip files
import { Reader, Writer } from '@transcend-io/conflux';

Creating a ZIP

Example using ReadableStream#pipeThrough

import { Writer } from '@transcend-io/conflux';
import streamSaver from 'streamsaver';

const s3 = 'https://s3-us-west-2.amazonaws.com/your-bucket/';
const files = ['NYT.txt', 'water.png', 'Earth.jpg'].values();

const myReadable = new ReadableStream({
  async pull(controller) {
    const { done, value } = files.next();
    if (done) return controller.close();
    const { body } = await fetch(s3 + value);
    return controller.enqueue({
      name: `/${value}`,
      stream: () => body,
    });
  },
});

myReadable
  .pipeThrough(new Writer())
  .pipeTo(streamSaver.createWriteStream('conflux.zip'));

// optionally, you can pass in a [queueing strategy](https://developer.mozilla.org/en-US/docs/Web/API/TransformStream/TransformStream#writablestrategy)
// to the constructure in order to specify the number of streams being consumed at a time
// default queue size is one, meaning only a single stream will be processed at a time
myReadable
  .pipeThrough(
    new Writer({
      // Write stream will allow 5 chunks in the underlying queue.
      // Once the entry's stream returns `done`, a new entry will be pulled
      // from `myReadable`.
      highWaterMark: 5,
      // each "chunk" in the queue represents 1 out of the total 5 we set for our limit
      size: (_: ZipTransformerEntry) => 1,
   }))
  .pipeTo(streamSaver.createWriteStream('conflux.zip'));

Example using writer.write

import { Writer } from '@transcend-io/conflux';

import streamSaver from 'streamsaver';

// Set up conflux
const { readable, writable } = new Writer();
const writer = writable.getWriter();

// Set up streamsaver
const fileStream = streamSaver.createWriteStream('conflux.zip');

// Add a file
writer.write({
  name: '/cat.txt',
  lastModified: new Date(0),
  stream: () => new Response('mjau').body,
});

readable.pipeTo(fileStream);

writer.close();

Incorporating other streams

import { Writer } from '@transcend-io/conflux';
import streamSaver from 'streamsaver';

const { readable, writable } = new Writer();
const writer = writable.getWriter();
const reader = readable.getReader();

// Set up streamsaver
const fileStream = streamSaver.createWriteStream('conflux.zip');

(async () => {
  writer.write({
    name: '/cat.txt',
    lastModified: new Date(0),
    stream: () => new Response('mjau').body,
  });

  const imgStream = await fetch(
    'https://s3-us-west-2.amazonaws.com/your-bucket/Earth.jpg',
  ).then((r) => r.body);

  writer.write({
    name: '/Earth.jpg',
    lastModified: new Date(0),
    stream: () => imgStream,
  });

  readable.pipeTo(fileStream);

  writer.close();
})();

Reading ZIP files

import { Reader } from '@transcend-io/conflux';

fetch('https://cdn.jsdelivr.net/gh/Stuk/jszip/test/ref/deflate.zip').then(
  async (res) => {
    const zip = await res.blob();
    for await (const entry of Reader(zip)) {
      console.log(entry);
    }
  },
);

Supporting Legacy Browsers

Conflux is compatible with all modern browsers since June 2022.

If you need to support legacy browsers, you can add polyfills for:

• TransformStream, and WritableStream (available in browsers since June 2022) by adding web-streams-polyfill.
• BigInt (available in browsers since January 2020) by setting globalThis.JSBI equal to JSBI before importing Conflux.
• globalThis (available in browsers since January 2020) by adding a polyfill like this globalthis or manually setting a shim.

License