Crawler Packages

NodeJs crawling & scraping framework heavily inspired by Scrapy (Pyhton)

NodeJS based crawler for CKAN sites

Simple, flexible, delightful web crawler/spider package

PIXNET posts crawler for node.js

> pixiv-crawler is a reptile for a website which named pixiv

Simple, configurable and extensible webcrawler

Page crawler that uses puppeteer

Price monitoring library, leveraging the power of x-ray and nightmare

Remote web content crawler done right.

Node.js database proxy for distributed applications

Simple Web Crawler

Create a XML sitemap for your website.

Forked version of https://github.com/ciffard/node-simplecrawler.

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

Unofficial APIs for Socialblade.com website.

Site scraper for Congregator

This module will scrape metadata from the URL shared

Indonesian online news scraper for NodeJS

Analyzes license information for multiple node.js modules (package.json files) as part of your software project.