Crawler Packages

Lightweight robots.txt parsing component without any external dependencies for Node.js.

Automatically extracts structured information from webpages

Run robots to search for TV Shows and Movies subtitles using the biggest subtitle website from Brazil and the world. They try their best to search using filename, metadata or command line options, either using a CLI or inside your node project.

[WIP] This project is just a personal challenge and has no further use - use it with caution!

Crawler for the GBIF API

The unofficial HLTV Node.js API use proxy

SiteLint crawls all of your pages and find errors from the crawled pages

Curated pull out sources

SKRIPTO CLI is a command line interface which helps developers to create and run Skripto Tasks (automated serverless scripts).

Mikros CLI is a command line interface which helps developers to create and run Mikros.io tasks (automated serverless scripts).

Esta biblioteca tem o objetivo de juntar vários crawlers das plataformas de compra/venda de automóveis em Portugal (ex: Stand Virtual) numa só ferramenta. Os crawlers navegam plataformas, recolhem informação e exportam um JSON com os dados de todos os car

A SoundCloud Web Crawler for music downloading.

SyphonX is a tool that extracts data from HTML data, transforming it into JSON of any shape or size. It combines the power of CSS Selectors and jQuery, Regular Expressions, and Javascript into a declarative template format to elegantly solve the simplest

Scrapy Framework implemented by nodejs.

Nodejs library that provides an Api for obtaining the movies information from FlixHQ website.

Nodejs library that provides an Api for obtaining the movies information from HDTO website.

Nodejs library that provides an Api for obtaining the movies information from CineGO website.

Nodejs library that provides an Api for obtaining the movies information from FlixHQ website.

extract a web page text content

An easy to use CLI for downloading websites for offline usage