web-csv-toolbox v0.11.0

🌐 web-csv-toolbox 🧰

A CSV Toolbox utilizing Web Standard APIs.

🔗

Key Concepts ✨

• 🌐 Web Standards first.
  • Utilizing the Web Standards APIs, such as the Web Streams API.
• ❤️ TypeScript friendly & User friendly.
  • Fully typed and documented.
• 0️⃣ Zero dependencies.
  • Using only Web Standards APIs.
• 💪 Property-based testing.
  • Using fast-check and vitest.
• ✅ Cross-platform.
  • Works on browsers, Node.js, and Deno.

Key Features 📗

• 🌊 Efficient CSV Parsing with Streams
  • 💻 Leveraging the WHATWG Streams API and other Web APIs for seamless and efficient data processing.
• 🛑 AbortSignal and Timeout Support: Ensure your CSV processing is cancellable, including support for automatic timeouts.
  • ✋ Integrate with AbortController to manually cancel operations as needed.
  • ⏳ Use AbortSignal.timeout to automatically cancel operations that exceed a specified time limit.
• 🎨 Flexible Source Support
  • 🧩 Parse CSVs directly from strings, ReadableStreams, or Response objects.
• ⚙️ Advanced Parsing Options: Customize your experience with various delimiters and quotation marks.
  • 🔄 Defaults to , and " respectively.
• 💾 Specialized Binary CSV Parsing: Leverage Stream-based processing for versatility and strength.
  • 🔄 Flexible BOM handling.
  • 🗜️ Supports various compression formats.
  • 🔤 Charset specification for diverse encoding.
• 🚀 Using WebAssembly for High Performance: WebAssembly is used for high performance parsing. (Experimental)
  • 📦 WebAssembly is used for high performance parsing.
• 📦 Lightweight and Zero Dependencies: No external dependencies, only Web Standards APIs.
• 📚 Fully Typed and Documented: Fully typed and documented with TypeDoc.

Installation 📥

With Package manager 📦

This package can then be installed using a package manager.

# Install with npm
$ npm install web-csv-toolbox
# Or Yarn
$ yarn add web-csv-toolbox
# Or pnpm
$ pnpm add web-csv-toolbox

From CDN (unpkg.com) 🌐

UMD Style 🔄

<script src="https://unpkg.com/web-csv-toolbox"></script>
<script>
const csv = `name,age
Alice,42
Bob,69`;

(async function () {
  for await (const record of CSV.parse(csv)) {
    console.log(record);
  }
})();
</script>

ESModule Style 📦

<script type="module">
import { parse } from 'https://unpkg.com/web-csv-toolbox?module';

const csv = `name,age
Alice,42
Bob,69`;

for await (const record of parse(csv)) {
  console.log(record);
}
</script>

Deno 🦕

You can install and use the package by specifying the following:

import { parse } from "npm:web-csv-toolbox";

Usage 📘

Parsing CSV files from strings

import { parse } from 'web-csv-toolbox';

const csv = `name,age
Alice,42
Bob,69`;

for await (const record of parse(csv)) {
  console.log(record);
}
// Prints:
// { name: 'Alice', age: '42' }
// { name: 'Bob', age: '69' }

Parsing CSV files from ReadableStreams

import { parse } from 'web-csv-toolbox';

const csv = `name,age
Alice,42
Bob,69`;

const stream = new ReadableStream({
  start(controller) {
    controller.enqueue(csv);
    controller.close();
  },
});

for await (const record of parse(stream)) {
  console.log(record);
}
// Prints:
// { name: 'Alice', age: '42' }
// { name: 'Bob', age: '69' }

Parsing CSV files from Response objects

import { parse } from 'web-csv-toolbox';

const response = await fetch('https://example.com/data.csv');

for await (const record of parse(response)) {
  console.log(record);
}
// Prints:
// { name: 'Alice', age: '42' }
// { name: 'Bob', age: '69' }

Parsing CSV files with different delimiters and quotation characters

import { parse } from 'web-csv-toolbox';

const csv = `name\tage
Alice\t42
Bob\t69`;

for await (const record of parse(csv, { delimiter: '\t' })) {
  console.log(record);
}
// Prints:
// { name: 'Alice', age: '42' }
// { name: 'Bob', age: '69' }

Parsing CSV files with headers

import { parse } from 'web-csv-toolbox';

const csv = `Alice,42
Bob,69`;

for await (const record of parse(csv, { headers: ['name', 'age'] })) {
  console.log(record);
}
// Prints:
// { name: 'Alice', age: '42' }
// { name: 'Bob', age: '69' }

AbortSignal / AbortController Support

Support for AbortSignal / AbortController, enabling you to cancel ongoing asynchronous CSV processing tasks.

This feature is useful for scenarios where processing needs to be halted, such as when a user navigates away from the page or other conditions that require stopping the task early.

Example Use Case: Abort with user action

import { parse } from 'web-csv-toolbox';

const controller = new AbortController();
const csv = "name,age\nAlice,30\nBob,25";

try {
  // Parse the CSV data then pass the AbortSignal to the parse function
  for await (const record of parse(csv, { signal: controller.signal })) {
    console.log(record);
  }
} catch (error) {
  if (error instanceof DOMException && error.name === 'AbortError') {
     // The CSV processing was aborted by the user
    console.log('CSV processing was aborted by the user.');
  } else {
    // An error occurred during CSV processing
    console.error('An error occurred:', error);
  }
}

// Some abort logic, like a cancel button
document.getElementById('cancel-button')
  .addEventListener('click', () => {
    controller.abort();
  });

Example Use Case: Abort with timeout

import { parse } from 'web-csv-toolbox';

// Set up a timeout of 5 seconds (5000 milliseconds)
const signal = AbortSignal.timeout(5000);

const csv = "name,age\nAlice,30\nBob,25";

try {
  // Pass the AbortSignal to the parse function
  const result = await parse.toArray(csv, { signal });
  console.log(result);
} catch (error) {
  if (error instanceof DOMException && error.name === 'TimeoutError') {
    // Handle the case where the processing was aborted due to timeout
    console.log('CSV processing was aborted due to timeout.');
  } else {
    // Handle other errors
    console.error('An error occurred during CSV processing:', error);
  }
}

Supported Runtimes 💻

Works on Node.js

Works on Browser

* To Be Tested: I couldn't launch Safari in headless mode on GitHub Actions, so I couldn't verify it, but it probably works.

Others

• Verify that JavaScript is executable on the Deno.

APIs 🧑‍💻

High-level APIs 🚀

These APIs are designed for Simplicity and Ease of Use, providing an intuitive and straightforward experience for users.

• function parse(input[, options]): AsyncIterableIterator<CSVRecord>: 📑
  • Parses various CSV input formats into an asynchronous iterable of records.
• function parse.toArray(input[, options]): Promise<CSVRecord[]>: 📑
  • Parses CSV input into an array of records, ideal for smaller data sets.

The input paramater can be a string, a ReadableStream of strings or Uint8Arrays, or a Uint8Array object, or a ArrayBuffer object, or a Response object.

Middle-level APIs 🧱

These APIs are optimized for Enhanced Performance and Control, catering to users who need more detailed and fine-tuned functionality.

• function parseString(string[, options]): 📑
  • Efficient parsing of CSV strings.
• function parseBinary(buffer[, options]): 📑
  • Parse CSV Binary of ArrayBuffer or Uint8Array.
• function parseResponse(response[, options]): 📑
  • Customized parsing directly from Response objects.
• function parseStream(stream[, options]): 📑
  • Stream-based parsing for larger or continuous data.
• function parseStringStream(stream[, options]): 📑
  • Combines string-based parsing with stream processing.
• function parseUint8ArrayStream(stream[, options]): 📑
  • Parses binary streams with precise control over data types.

Low-level APIs ⚙️

These APIs are built for Advanced Customization and Pipeline Design, ideal for developers looking for in-depth control and flexibility.

• class LexerTransformer: 📑
  • A TransformStream class for lexical analysis of CSV data.
• class RecordAssemblerTransformer: 📑
  • Handles the assembly of parsed data into records.

Experimental APIs 🧪

These APIs are experimental and may change in the future.

Parsing using WebAssembly for high performance.

You can use WebAssembly to parse CSV data for high performance.

• Parsing with WebAssembly is faster than parsing with JavaScript, but it takes time to load the WebAssembly module.
• Supports only UTF-8 encoding csv data.
• Quotation characters are only ". (Double quotation mark)
  • If you pass a different character, it will throw an error.

import { loadWASM, parseStringWASM } from "web-csv-toolbox";

// load WebAssembly module
await loadWASM();

const csv = "a,b,c\n1,2,3";

// parse CSV string
const result = parseStringToArraySyncWASM(csv);
console.log(result);
// Prints:
// [{ a: "1", b: "2", c: "3" }]

• function loadWASM(): Promise<void>: 📑
  • Loads the WebAssembly module.
• function parseStringToArraySyncWASM(string[, options]): CSVRecord[]: 📑
  • Parses CSV strings into an array of records.

Options Configuration 🛠️

Common Options ⚙️

Advanced Options (Binary-Specific) 🧬

How to Contribute 💪

Star ⭐

The easiest way to contribute is to use the library and star repository.

Questions 💭

Feel free to ask questions on GitHub Discussions.

Report bugs / request additional features 💡

Please register at GitHub Issues.

Financial Support 💸

Please support kamiazya.

Even just a dollar is enough motivation to develop 😊

License ⚖️

This software is released under the MIT License, see LICENSE.