Crawler Packages

A library of functions for web crawling using BFS.

crawler toolbox

Crawl websites and convert them to JSON with ease

Crawlyx is an open-source command-line interface (CLI) based web crawler built using Node.js. It is designed to crawl websites and extract useful information like links, images, and text. It is lightweight, fast, and easy to use.

Nexstack Nodejs library that provides an Api for obtaining the movies information website.

Easily create a scraper api with the @web/scrapper library, which includes a scraper and advanced events for your website.

A twitter client for agents

A set of Web Scraping robots to get Dragon City data from websites.

Distributed web crawler powered by Headless Chrome

一个专门用于爬取csdn文章的爬虫/A JS library for Crawl CSDN Article.

The unofficial HLTV Node.js API by (https://www.linkedin.com/in/lucasc0rrea/)

Simple & Human-Friendly HTML Scraper with Json-ld support

A TypeScript library for detecting and categorizing bots from user agent strings

The directory crawler library for Node.JS

Dex8 CLI is a command line interface which helps developers to create and run dex8.com skripts (automated serverless scripts).

DEX8 Library used in DEX8 platform to build DEX8 robot tasks.

DEX8 SDK is a library which helps developers to create DEX8 crawler, scraper tasks or automated serverless scripts.

基于puppeteer的爬虫

From the Bay to LA, scraper will collect all of the images from the url you tell it to and return a list of the images that best represent the site.

This is a client for the Elastic Web Crawler API