Scraping Packages

Really simple HTTP client. Mainly for scraping but not only.

A web scraping tool that extracts any data from the web.

Natural-language web automation using Puppeteer

a scraper multitool

Scrape URL with CSS selectors

Unofficial APIs for Socialblade.com website.

Node.js package to bypass WAF anti-bot JavaScript challenges

Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously

Converts all websites to JSON data

Plugin based node.js web scraper. It scrapes, stores and exports data. Supports multiple storage options: SQLite, MySQL, PostgreSQL. Supports multiple browser or dom-like clients: Puppeteer, Playwright, Cheerio, Jsdom.

A crawler based on Phantom. Allows discovery of dynamic content and supports custom scrapers.

A very easy-to-use and lightweight web scrapper

Extract links from Google SERP

Real transparent HTTP-Proxy-Server. Upstream your requests whatever you want!

A module using puppeteer to scrape several search engines such as Google, Bing and Duckduckgo

An automation package

Scrape image urls from HTML website including CSS background images.

Scraper for oEmbed, Twitter Cards and Open Graph metadata - fast and Promise-based

Scraping comments from Youtube.

DCrawler is a distribited web spider written in Nodejs and queued with Mongodb. It gives you the full power of jQuery to parse big pages as they are downloaded, asynchronously. Simplifying distributed crawler!