Crawler Packages

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Crawlora SDK for interacting with the Crawlora platform.

Crawlora SDK for interacting with the Crawlora platform.

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Templates for the crawlee projects

A port of n0madic/twitter-scraper to Node.js.

Auxiliary package of functions for the TypeScript framework Botmation focused on Facebook

Auxiliary package of functions for the TypeScript framework Botmation focused on Reddit

Auxiliary package of functions for the TypeScript framework Botmation

blocklet crawler lib

A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.

Endless Crawler Cached Data

irasutoya crawler cli

A crawler to fetch last news from Iranian(Persian) news agencies.

Simple news ticker based on Scott Alguire jQuery grouploop plugins

Async and sync crawler for json object

This library provides support for traversing objects and their values while providing information on the traversal state, pathing to target values, and the ability to manipulate said pathing to easily move to related values.

crawler

A middleware used to determine the req whether it is a browser bot.