simplecrawler
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
Priority based Semantic Web Crawler.
Very straigntforward web crawler. Uses EventEmitter. Generates queue statistics and has a basic cache mechanism with extensible backend. This version is forked in order to add the ability to filter by referrer url
Very straightforward web crawler. Uses EventEmitter. Based on Simplecrawler but using a distributed queue system.
Forked version of https://github.com/ciffard/node-simplecrawler.
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
iSpider is a simple and fast web spider written with Nodejs!
MongoDB queue implementation for simplecrawler
A web crawler forked from simplecrawler.
web-spider is a simple and fast web spider written with Nodejs!
SQLite FetchQueue Implementation for Simplecrawler
A web crawler forked from simplecrawler.
MongoDB FetchQueue Implementation for Simplecrawler
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.