crawler-url-parser
An `URL` parser for crawling purpose.
An `URL` parser for crawling purpose.
Url scraper which takes the text input and finds the links/urls, scraps them using cheerio and will returns an object with original text, parsed text (using npm-text-parser) and array of objects where each object contains scraped webpage's information.
The module can parse most urls into different usable parts