Crawler Packages

A library for efficiently walking a directory recursively

A JavaScript libary to easily use SpeedyShot's capture service

The [spider](https://github.com/spider-rs/spider) project ported to Node.js

The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

A Bing command line dictionary, which obtains the query results of bing dictionary by crawler.

The unofficial HLTV Node.js API

PhantomJS/Browser lib which allows to parse a webpage

A CLI tool to crawl (for example) documentation websites and convert them to Markdown.

🤖 Fake fingerprints to bypass anti-bot systems. Simulate mouse and keyboard operations to make behavior like a real person.

JavaScript SDK for Firecrawl API

JavaScript SDK for Firecrawl API

A robust web scraping library using Playwright for Node.js. This library provides an easy-to-use API for automating web interactions, extracting data, and handling various web scraping tasks efficiently.

Distributed web crawler powered by Headless Chrome

The server crawler for Rabbit Server List

Easily create XML sitemaps for your website with url-parse 1.5.10 based on sitemap generator

Crawler for Gerrit

A library for crawling a filesystem tree, based on Effect-ts

JavaScript SDK for Spidery API

Model Context Protocol (MCP) server for pure.md, the markdown delivery network for LLMs

MCP server for Firecrawl web scraping integration. Supports both cloud and self-hosted instances. Features include web scraping, batch processing, structured data extraction, and LLM-powered content analysis.