Collection of patches for puppeteer and playwright to avoid automation detection and leaks. Helps to avoid Cloudflare and DataDome CAPTCHA pages. Easy to patch/unpatch, can be enabled/disabled on demand.
A tool to get sitemaps from websites and crawl them
Gracefully handle timeout and network error with auto retry.
Distributed web crawler powered by Headless Chrome
A web-crawler and scraper that extracts data from a family of nested dynamic webpages with added enhancements to assist in knowledge mining applications.
A lightweight and simple API for web crawling built on chromium puppeteer
A set of shared utilities that can be used by crawlers
Web crawler for Node.js
Node.js client for the CloudCrawler.io API
Dependency free module for scraping and crawling websites using [Crawlbase](https://crawlbase.com) API
A simple crawler made in JavaScript for Node.
Transform your text with dynamic typing animations! crawling-typer lets you display an array of strings one at a time, each with its own color. Customize typing speed, delete speed, and pauses between strings. Enjoy full control with loop counts, post-loo
Crawlyx is an open-source command-line interface (CLI) based web crawler built using Node.js. It is designed to crawl websites and extract useful information like links, images, and text. It is lightweight, fast, and easy to use.
A lightweight and modular web crawling framework built with Puppeteer.
Easily create a scraper api with the @web/scrapper library, which includes a scraper and advanced events for your website.
This is the React Component for Detect Crawling
Distributed web crawler powered by Headless Chrome
Simple & Human-Friendly HTML Scraper with Json-ld support
Tem o objetivo de executar rotinas de CRAWLING a partir de um arquivo JSON utilizando xpath mas aceitando para cada passo uma função callback que recebe o valor e pode passar esse valor para um próximo passo.
A Node.js scraping framework built on puppeteer (to use a headless Chrome/Chromium browser)