Crawler Packages

Facebook Spider CORE

Facebook Spider CLI

Crawls the file system and streams the directory/file information

A simple crawler or scraper to get open graph and other meta data from any website.

crawl the fritter network

CLI tools for SourceCode

This plugin links your Netlify site with Algolia's Crawler. It will trigger a crawl on each successful build.

Module to retrieve ps5 pre-order and availability status

Distributed web crawler powered by Headless Chrome

Library packages used by Lightnovel Factory (https://github.com/dipu-bd/lightnovel-factory)

Get product details as a json reponse including feedbacks, variants, description, images, etc.,

The unofficial HLTV Node.js API

Forked package

A util tool

用于爬取环球网的新闻数据

crawl youtube without api key (search videos, search channels, get all channel's videos)

MrCrawler is a simple but powerful crawler engine that you can use to broad crawl with no eforce. MrCrawler is built on top of Puppeteer and Playwright. You can also use Redis with a built in worker distribuiter allowing you use MrCrawler concurrently in

Node.js agent for Sqreen, please see https://www.sqreen.io/

Migrations manager for mongoose

automatically send message to friends at linkedin