crawl_fetcher
A web crawler. Automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.
1.0.0 • Published 4 years ago
Spider Packages
crawlab-node-sdk
sdk for crawlab nodejs runtime add data to result collection
crawler-lian
A configuration - based crawler framework
book-spider
a tool for downloading book
dataspider
数据挖掘机
exclusion
A simple, RFC-compliant robots.txt parser
enispider
A Node.js scraping framework built on puppeteer (to use a headless Chrome/Chromium browser)
farmer-site-cli
站长工具，爬取网站网页数据，命令行式操作，无需代码编写。
fast-spider
A simple and fast crawler tool based on node
flash-spider
闪电般的爬图软件
freeproxylists-scrapper
Page scrapper that retrieves proxy server information from FreeProxyLists website
@vpukhanov/quick-scraper
An easy, lightweight scraper for humans with many inbuilt features..
@yunyoujun/douban-spider
豆瓣爬虫小工具 By Node.js.
@youmoo/generic-crawler
A generic crawler interface
@youlingred/spider.io
极简网络蜘蛛爬虫，适用任何网站，只需设置一条规则，就可以把你想要网站上的内容整理出来，非常方便，简单！
acfun-article-spider
acfun文章区爬虫工具
@zspider/core
ZSpider Core
@znode/find-chrome
Get Chrome Path for Puppeteer
@znode/browser
Create Browser with Puppeteer for Crawl
aliexpress-product-scraper-ts
Get Aliexpress product details as a json reponse including feedbacks, variants, description, images, etc.,