Spider Packages

Manages queues of requests for data processing and collection.

simple crawler

A aliexpress spider

Bittorrent dht network infohash spider, for engiy.com[a bittorrent resource search engine]

[META-PACKAGE] Let your users be your testers. Catch every error from your production JavaScript code, whether it's React, Vue, Cordova or pure and vanilla JS — JS-Spider got you covered!

Wechat Spider

Gathers the HTML content of a spec, a list of subresources it fetches and, optionally, a screenshot of the spec.

Web crawling and scraping engine.

A simple evented web scraping framework using node.js

Channels Library for Spider

a components for react-native

A 2nd generation spider to crawl any article site, automatic reading title and content.

CLI for Spiderman

A light-weight framework of spider implemented in node.js.

A spider test runner for your website

Very straightforward web crawler. Uses EventEmitter. Based on Simplecrawler but using a distributed queue system.

Spiderman makes it trivial work to write a crawler. Just define some components, use segues to connet them, finally write some script to specify data that needs to be transfer from one component to the next, that's all!

sipder network with nodejs

Robust web spider for Node.js

A crawler framework based on NodeJS.