0.0.6 • Published 4 years ago
web-crawler-scaffold v0.0.6
web-crawler-scaffold
封装 request、cheerio ,用来当 node 爬虫的脚手架
Installation
$ npm i -g npm
$ npm i --save web-crawler-scaffold
In Node.js
const Crawler = require('web-crawler-scaffold')
const crawler = new Crawler()
API
// 发送 Get 请求
const { res, body, $ } = crawler.triggerGet({ url, headers? })
// 发送 Post 请求
const { res, body, $ } = crawler.triggerPost({ url, headers?, formData? })
// 休眠
await crawler.sleep()
todo
- 引入 cheerio,解析 html
- 代理ip
- 引入 Puppeteer ?