Crawler Packages

Library to extractor all social (profile) links from a website

Lightweight crawler written in TypeScript using ES6 generators.

A website crawler. Stays in the same website and return every internal linked URL with the associated HTTP status code

A high-performance automatic crawler library

Thin wrapper around a compilation of common user agent strings that one can query easily through tags.

A nodejs crawler module

Crawl 100% JS single page apps with phantomjs and node.

The simple and fast crawling framework. So you can focus on scraping.

Yet another node torrent scraper based on x-ray. (Support iptorrents, torrentleech, torrent9, Yyggtorrent, ThePiratebay, torrentz2, 1337x, KickassTorrent, Rarbg, TorrentProject, Yts, Limetorrents, Eztv)

Distributed web crawler powered by Headless Chrome

google search crawler

A node library to crawler imdb and parse to json

A set of Gulp commands that provide similar functionality to Apache Nutch.

An easy to use CLI for downloading websites for offline usage

Fetch the pre-rendered content, meta, links and Open Graph of a webpage, especially Single-Page Application (SPA)

[](https://npmjs.org/package/typescript-mysql-model) [](https://david-dm.org/joelsmedberg/typescript

instagram crawler

Fetch Inha Univ. Information from Official Website

PhantomJS accessibility auditor

A highly configurable website crawler for automatically testing a website for accessibility issues using the axe-core library. Uses selenium and headless Chrome to load pages, inject axe-core, and run tests. Generates an html summary report in addition