Crawler Packages

Get cjdropshipping product details as a json reponse including feedbacks, variants, description, images, etc.,

A tiny node module to detect spiders/crawlers quickly and comes with optional middleware for ExpressJS

A bot to crawl impots.gouv.fr

A robust web scraping library using Playwright for Node.js. This library provides an easy-to-use API for automating web interactions, extracting data, and handling various web scraping tasks efficiently.

JavaScript SDK for Firecrawl API

JavaScript SDK for Firecrawl API

🤖 Fake fingerprints to bypass anti-bot systems. Simulate mouse and keyboard operations to make behavior like a real person.

A CLI tool to crawl (for example) documentation websites and convert them to Markdown.

PhantomJS/Browser lib which allows to parse a webpage

The unofficial HLTV Node.js API

Jacob's Crawler, the Library version.

Jacob's Crawler, the CLI version.

Scrape public available jobs on Linkedin using headless browser

Distributed web crawler powered by Headless Chrome

A simple license crawler for crediting open source work

Web data extraction can be effectively performed using CSS selectors.

A Node.js fetch tool that imitates browser behavior and ready for crawlers.

类似java中webmagic类似的爬虫框架

Detect user-agent is a bot/spider/crawler