0.1.1 • Published 9 years ago
sandcrawler-logger v0.1.1
sandcrawler-logger
Simple logger to plug into one of your sandcrawler spiders for immediate feedback.
Installation
You can install sandcrawler-logger through npm:
npm install sandcrawler-logger
Usage
var sandcrawler = require('sandcrawler'),
logger = require('sandcrawler-logger');
var spider = sandcrawler.spider('MyFancySpider')
.use(logger())
.url('http://nicesite.org')
.scraper(function($, done) {
done(null, $('title').text());
})
.run();
Options
- color ?string
'magenta'
: color attributed to your spider. This is useful when running several spiders at once when you want to clearly distinguish their respective logs. - level ?string
'debug'
: Threshold level for the log, order being:debug
,verbose
,info
,warn
anderror
. - pageLog ?boolean
true
: in case of a phantom spider, should we display the web page's log and errors?
Example
var sandcrawler = require('sandcrawler'),
logger = require('sandcrawler-logger');
var spider = sandcrawler.spider('MyFancySpider')
.use(logger({color: 'blue', level: 'warn'}));
Spider's logger
When plugging the logger onto your spider, this one will be attached a logger
property so you can push custom information through the means of the plugin if needed.
var sandcrawler = require('sandcrawler'),
logger = require('sandcrawler-logger');
var spider = sandcrawler.spider('MyFancySpider')
.use(logger())
(...)
.result(function(err, req, res) {
this.logger.info('Page title: ' + res.data);
})
.run();
Note that, under the hood, this plugin uses the winston library, on whose documentation you can find more information about how to use the spider.logger
property.
License
MIT