Ak-fetch NPM | npm.io

ak-fetch

tldr;

a "batch and queue" HTTP client for making bulk POST requests to an API endpoint:

🌍 Overview

ak-fetch is a powerful and flexible tool designed for making bulk POST requests to an API endpoint. It provides a simple interface to batch and queue requests which send data over the network; it adds oft needed features like concurrency control, retries, fire-and-forget making it a robust solution for handling large data transfer operations with ease.

i built this module when creating hello-mixpanel as i needed a way to send large amounts of data to various SaaS analytics APIs in a reliable and efficient manner. it was useful enough for me, that i made it into a proper package.

🚀 Installation

To get started with ak-fetch, install the module using npm:

npm install ak-fetch

you may also use npx to run the CLI without installing the package:

npx ak-fetch --help

🖥️ Usage

Use ak-fetch as a module directly in your node script:

const akFetch = require('ak-fetch');
const config = {
        url: 'https://api.example.com',
        data: [...],
		batchSize: 10,
		concurrency: 5,
		delay: 1000,
        // ... other configurations
};

try {
	const responses = await akFetch(config);
	console.log('API Responses:', responses);
} catch (error) {
	console.error('Error:', error);
}

or via the command line:

npx ak-fetch --url https://api.example.com './payloads.json' --batchSize 10 --concurrency 5

Use --help to see all options

🛠️ Configuration

The ak-fetch module can be configured with a variety of options to suit your needs:

Option	Type	Description
`url`	`string`	The URL of the API endpoint.
`data`	`Object[]`	An array of data objects to be sent in the requests.
`batchSize`	`number`	# records per batch; Use 0 for no batching.
`concurrency`	`number`	The level of concurrency for the requests.
`delay`	`number`	The delay between requests in milliseconds.
`searchParams`	`Object`	The search parameters to be appended to the URL.
`bodyParams`	`Object`	The body parameters to be sent in the request.
`headers`	`Object`	The headers to be sent in the request.
`verbose`	`boolean`	Whether to log progress of the requests.
`dryRun`	`boolean`	If true, no actual requests are made.
`logFile`	`string`	File path where responses will be saved.
`retries`	`number`	Number of retries for failed requests; use `null` for fire-and-forget.
`retryDelay`	`number`	Delay between retries in milliseconds.
`retryOn`	`number[]`	HTTP status codes to retry on.