Crawler Packages

Search for anything on web.

Device detection on-premise services for the 51Degrees Pipeline API

Device detection cloud services for the 51Degrees Pipeline API

Shared functionality for implementing device detection engine for the 51Degrees Pipeline API

Ядро веб-краулера для обхода сайта

bot to download pictures from imagozone.com

The unofficial HLTV Node.js API

A node.js wrapper for the PageMunch link unfurling API

Bogeyman is application build upon awesome PhantomJS and it provides REST API so you can use PhantomJS headless crawling of heavy javascript webspages within any programming language or curl.

Scrap the web asynchronously in live, reusing Node.js, all in one file, with a few lines!

Nodejs Distribute Crawler

Environment for Goose parser which allows to run it in commmon Browser

Memory efficient and synchronous downloader of map tiles. Allows for a fast and easy approach to make map tiles (from a WMS) available offline.

* Install the package globally.

bot to download pictures from theplace2.ru

A full-fledged node.js web crawler with a MySQL backend

Crawler made simple

Uses cheerio to return a structured JSON representation of a Wikipedia article.

Boilerplate code for a Node.js scraper with CLI

Daily use crawling methods for puppeteer