scraper_bike v1.0.10

Scraper Bike

Run this small program against a set of urls contains in a .txt file with a set of strings provided as arguments to see whether the html of those pages contain those strings.

Requirements:

>= Node.js 5.10

Installation:

npm install -g scraper_bike

or from source; clone this repo and run:

node bin/scraper_bike.js [options]

Example Usage:

our targets.txt file would look like this:

http://www.bankofamerica.com
http://fidelity.com
http://ally.com

We can run:

$ scraper_bike targets.txt '<meta name="keywords"' 'commission'

Outputs:

[403 Forbidden], http://ally.com
http://ally.com, false, <meta name="keywords", commission
http://fidelity.com, true, <meta name="keywords", commission
http://www.bankofamerica.com, false, <meta name="keywords", commission