concurrentfetcher v1.2.2

ConcurrentFetcher

A javascript class for managing concurrent fetch requests.

The Fetch Web API is a neat tool for fetching/getting network resources in web applications. And although fetch() is generally easy to use, there are a few nuances in regards to error handling, asynchronous and concurrent processing, cancellation and so forth :-)

The ConcurrentFetcher class addresses the core challenges of concurrent requests, error handling, and cancellation.

Availability

Maintained at github : https://github.com/asicscreed/ConcurrentFetcher And published at npm: https://www.npmjs.com/package/concurrentfetcher

Install

npm install concurrentfetcher

Usage

Basically, you instantiate the class with an array of fetch requests and then call concurrentFetch(). For browsers it must be in an async context. It calls fetch and consumes the Response object. If a callback is defined, it is called for each response. Without a callback, the responses (data and errors, respectively) are collected and returned.

Browser example without callback (src="concurrentfetcher.iife.min.js"):

JavaScript: Example1

const requests = [{ url: "https://jsonplaceholder.typicode.com/photos/1" }, { url: "https://jsonplaceholder.typicode.com/comments/1" }];

const fetcher = new ConcurrentFetcher.ConcurrentFetcher(requests);
fetcher.concurrentFetch()
.then((data) => {
  const errors = data.errors ?? {};
  const results = data.results ?? {};
  if (errors.length > 0) document.write(JSON.stringify(errors));
  if (results.length > 0) document.write(JSON.stringify(results));
})

When data is returned without callback (as above), then data is a Promise<ConcurrentFetchResults> as this:

interface ConcurrentFetchResult {
  results: any[]; // json, text or blob data
  errors: { uniqueId: string; url: string | Request; error: FetchError }[];
}

The errors array is added with the unique Id, the request URL and a custom FetchError which sets error.status as response.status. The result array contains what is in the Response objects

Same currentFetch example, but with callback (src="concurrentfetcher.iife.min.js"):

JavaScript: Example2

const requests = [
  { url: "https://jsonplaceholder.typicode.com/photos/1",
    callback: (uniqueId, data, error, abortManager) => {
      if (error) document.write(JSON.stringify(error));
      else document.write(JSON.stringify(data));
    }
  },
  { url: "https://jsonplaceholder.typicode.com/comments/1",
    callback: (uniqueId, data, error, abortManager) => {
      if (error) document.write(JSON.stringify(error));
      else document.write(JSON.stringify(data));
    }
  }
];

const fetcher = new ConcurrentFetcher.ConcurrentFetcher(requests);
fetcher.concurrentFetch()
...

Hosted in Node.js with: concurrentfetcher.umd.min.js

JavaScript: Node.js

const people  = document.getElementById('people');
const fetcher = new ConcurrentFetcher.ConcurrentFetcher(requests);
fetcher.concurrentFetch()
.then((data) => {
  // you should handle if (data.errors.length > 0)
  if (data.results.length > 0) {
    people.innerHTML = data.results[0].map((person) => {
      return (
        '<div class="text-center mt-3">'+
          '<h5 class="mt-2 mb-0">'+person.name+'</h5>'+
          '<span>'+person.email+'</span>'+
          '<div class="px-4 mt-1">'+
            '<p class="fonts">'+person.company.catchPhrase+'</p>'+
          '</div>'+
        '</div>'
      );
    }).join('');
  } // data.results.length
})
.catch(error => console.error(error));  

Loaded in RequireJS with: concurrentfetcher.amd.min.js

JavaScript: RequireJS

requirejs.config({ paths: { ConcurrentFetcher: '/concurrentfetcher.amd.min' }});
requirejs(['ConcurrentFetcher'], function (ConcurrentFetcher) {
  const requests = [
      {
          url: 'https://api.github.com/users/asicscreed',
          headers: {
            Accept: "application/json",
            "Content-Type": "application/json",
          },
      }
  ];
  const fetcher = new ConcurrentFetcher.ConcurrentFetcher(requests);
  fetcher.concurrentFetch()
  .then(function (data) {
      if (data.results.length > 0) {
          user = data.results[0];
          document.getElementById('useravatar').src = user.avatar_url;
          document.getElementById('username').innerHTML = user.login;
      }
  })

Documentation:

Currently, all documentation is located in the docs folder. It is generated by JSDoc, and is therefore regular html documents.

Key points:

Concurrency: The class leverages Promise.all() for efficient concurrent execution and optimizing performance. Error handling: Custom error classes (FetchError, JsonParseError) and error reporting provide information for debugging and handling failures. Flexibility: The ability to configure individual fetch options and utilize both callbacks and promises makes the class adaptable to various use cases. Cancellation: The AbortManager class provides robust cancellation support, allowing for individual and global request cancellation. Support for client-controlled cancellation of a single request and of all requests. And timeout controlled cancellation on individual requests. Progress tracking: The optional progressCallback enables monitoring the progress of fetch requests. Retry logic: Retry mechanism for failed requests to improve the resilience of the class. Large data handling: Utilizes response.body to read large data in chunks.

Areas for consideration:

Advanced progress Tracking: Implement more granular progress tracking, such as byte transfer information for more detailed monitoring. Stream handling: Adding support for handling response streams, which would be beneficial for large data transfers. Testing: (More) Extensive unit and integration tests will greatly improve the classes reliability. Adaptability: Examples to demonstrate how to use the class in various environments (browser, Node.js, frontend frameworks, testing).

Get help: Post in our discussion board • Review the GitHub status page

© 2023 GitHub • Code of Conduct • MIT License