# rakes

> web scraper

Latest version **0.0.8** (published 2015-09-04) · MIT license · 0 weekly downloads

## Install

```sh
npm install rakes
pnpm add rakes
yarn add rakes
bun add rakes
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support; pre 1.0.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 0.0.8 |
| Published | 2015-09-04 |
| First published | 2014-11-06 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 2 |
| Known vulnerabilities | 0 (+1 in 1 direct dependencies) |
| Install scripts | no |
| Author | Will Dunning |
| Maintainers | willdunning |

## Links

- npm: https://www.npmjs.com/package/rakes
- Repository: https://github.com/willdunning/rake
- Issues: https://github.com/willdunning/rake/issues
- npm.io page: https://npm.io/package/rakes

## Dependencies (2)

- [cheerio](https://npm.io/package/cheerio.md) ^0.15.0
- [request](https://npm.io/package/request.md) ^2.34.0

## Recent versions

- 0.0.8 (latest) — 2015-09-04
- 0.0.7 — 2015-09-04
- 0.0.6 — 2014-11-07
- 0.0.5 — 2014-11-07
- 0.0.4 — 2014-11-07
- 0.0.3 — 2014-11-06
- 0.0.2 — 2014-11-06
- 0.0.1 — 2014-11-06

## README

#rakes

rakes is a web scraper for Node.JS

## Version
0.0.4

## Installation

Install locally (for now):
```sh
$ npm install rakes
```

## Setup

#### Sites.json
Configuration of sites to  be crawled.

Required Values:
 - url : url of site
 - id : unique id within this json
 - scraper : path to scraper relative to rakes.js 
 
Optional Values:
- iterator : if iterating over sites 
- limit : index where iterator stops
- startIndex : index where iterator starts

EX :
```
[     
      {
         "url":"http://www.craigslist.org/about/sites",
         "id":"cl",
         "scraper":"../../scrapers/cl"
      }, 
      {
         "url":"https://news.ycombinator.com/news?p=",
         "iterator":"1",
         "limit":"5",
         "startIndex":"1",
         "id":"yc",
         "scraper":"../../scrapers/yc"
      }
]
```


###Scrapers

Scrapers are modules that are used to parse the html from individual entries in your sites.json

Parameters
    - html - html from response
    - app - reference to rakes
    - cheerio - lets us use jquery like syntax

Public Properties:
    'app.scrapeData' : The params property that holds the collective scraped data
    
Events:
    - 'scraperComplete' : to be dispatched when scraper has fully parsed html.


EX:
```
module.exports.scraper = function(html,app,cheerio) {
	var $ = cheerio.load(html);
	$('td.title').each(function(){	
		var el = $(this);
		var title = el.text();
		app.scrapeData.push(title);
	})
	app.pubsub.emit('scraperComplete')
}
```

###Instantiation
Rake(sites.json , callback) 

```
var express = require('express');
var sites = require('./sites');  
var rakes = require('./rakes');
var app = express();
var PORT = process.env.PORT || '8081';

app.listen(PORT)
exports = module.exports = app;
 
rakes.Rake(sites,function(data){
	console.log(data)
	process.kill();
});
```



### Dependencies
* Cheerio
* Request
 
### Todo's

 - fix global install
 - get a haircut

License
----

MIT

---
_Source: https://npm.io/package/rakes · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._