Crawler Packages

promise based interface to inject a script into a phantom browser and resolve a result

Lightweight crawler that works like a real browser

Web crawler test script

A web spider app, it converts a page into a json object

configurable API crawler for scraping feeds to static files

An intelligent proxy and request manager.

crawling data from website by nodejs

Manga downloader, uses [gin-downloader](https://github.com/pikax/gin-downloader) as a engine.

nodejs modules for fetching github trending data

transform github-trending repos to json data

A simple command line app for controlling a GitHub Crawler

Crawler UI Dashboard

Crawl info from one git account

use google play protobuf api in node

use google play protobuf api in node

Express middleware that returns the resulting html after executing javascript, allowing crawlers to read on the page

Express middleware that returns the resulting html after executing javascript, allowing crawlers to read on the page

nodejs modules used for fetching github trending data

Given a username, GIC will crawl through their repositories grabbing all issues and outputting them to a local folder.

Given a username, GIP will crawl through their repositories grabbing all issues and outputting them to a local folder as PDFs.