Crawler Packages

Lightweight crawler written in TypeScript using ES6 generators.

Lightweight crawler written in TypeScript using ES6 generators.

A crawler that crawls (no surprise there) websites and checks wether the copyright year is up-to-date or not. #OCD

A search and crawler for Wikipedia articles

A web crawler written in TypeScript.

Pluggable, flexible and agnostic prerenderer for sites and SPAs, optimized for SEO. No code changes required.

tiny-crawler is a web crawler.

The core code for crawler

Unofficial API for zhihu (https://www.zhihu.com)

Crawl all urls of a domain

A crawler for a dhis2 instance to pull data from the site.

A painless Node.js web crawler that simply works

From the Bay to LA, scraper will collect all of the images from the url you tell it to and return a list of the images that best represent the site.

Turns web pages into an rss feeds

Unofficial API lib for Nintendo Switch eShop game listing and pricing information.

Module to track all npm dependencies and their corrosponding licenses.

Protect text or link against web crawlers / bots

a simple node crawler

Analyzes license information for multiple node.js modules (package.json files) as part of your software project.

A web page content extractor for News websites