@raing3/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
1.0.1 • Published 12 months ago
Webcrawler Packages
@qbbsh/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
@public-square/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
@rodrigogs/spiderman
Yet to describe
myreadingmanga-unofficial
MyReadingManga WebCrawler. Find By Languages, Genres and Tags.
schabbi-webscraper
Lightweight and easy to use crawling solution for websites.
redis-web-crawler
Web Crawler to create directed graph of links among connected sites. Runs with Node.js and stores data with Redis
readme-crawler
Download README files from GitHub repository links
sitemapper_mos
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
htmlgetty
This is a function that accepts 3 arguments, "url", "tag" and "output", and writes to a file, in the "output" path, the content of an html "tag", relative to a specific "url".
@yeskiy/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
adibenmat-sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers
crawl-cli
A Node crawler/scrape for retrieving data from websites