Crawler Packages

Promise support for node-crawler (Web Crawler/Spider for NodeJS + server-side jQuery)

Easily create XML sitemaps for your website.

CLI tool for SEO, scans one or several sites and generate CSV and/or XLSX

This module is propose to scrapy website pages and extract information from doms which selected by jQuery-like selectors

Simple manga scrapper for famous online manga websites.

grabbi a simple web scraper/crawler

Tool for easy scraping data from websites

A news scraper for javascript that help to scrap news from Indonesian news portal.

Reactive, configurable web crawler

fork from headless-chrome-crawler and update puppeteer to the latest version

node js command for check insecure content in whole link in your website

The unofficial HLTV Node.js API

A node library to gets Movie and Tv Series info from IMDB and returns collected data as a json

Easy To Use Web Crawler

Hapi Crawler plugin help you detect bots/crawlers/spiders via the useragent.

Simple Instagram Crawling without using public API

Listing crawler model for crawlers and tools

a simple recursive file crawler

Crawl the log of any command

Convert a website to static markdown.