mrspider v5.1.2

Mr Spider

Crawl the web politely.

Installation

$ npm i mrspider --save

Example

An example for parking data

Middleware

####Fetch the page

mrspider request

####Parse DOM

mrspider cheerio

mrspider JSDOM

####Parse Data

mrspider regex data extractor

mrspider css data extractor

mrspider css links

mrspoder image extraction

####Data validation

mrspider validator

####Data persistence

mrspider mongodb persister

Features

• Super simple api.
• Streaming architecture allows complete customisation.
• Use the full power of JavaScript giving you great flexibility.

Tests

To run the test suite, first install the dependencies, then run npm test: