Spider Packages

Environment for Goose parser which allows to run it in commmon Browser

Create a spider for spiderman

A full-fledged node.js web crawler with a MySQL backend

Crawler made simple

[Browser Package] Let your users be your testers. Catch every error from your production JavaScript code, whether it's React, Vue, Cordova or pure and vanilla JS — JS-Spider got you covered!

puppeteer crawler

Very straigntforward web crawler. Uses EventEmitter. Generates queue statistics and has a basic cache mechanism with extensible backend. This version is forked in order to add the ability to filter by referrer url

a reliable web crawling & scraping framework for Node.js.

爬虫工具,根据传入的配置和规则爬取数据

A simple web crawler that keeps to the domain

Crawl bbb.org for leads

🕷️ Adapter for spiders, using Cheerio to parse HTML response.

simple polite crawling of the web.

Paginator enriches ability to paginate over the pages in Goose Parser

And ExpressJS middleware for detecting search engine crawlers/spiders and handling crawler-specific logic such as additional logging for requests from spiders

Parser for Filmaffinity site

Web spider and take screenshot for input URL. Only spider the same domain, under the same pathname.

fetch html for spa web application

spdier request

A @0y0/scraper expansion pack.