the-crawl v1.0.0

the-crawl

Crawler for the-frameworks

Installation

$ npm install the-crawl --save

Usage

'use strict'

const { TheCrawl } = require('the-crawl')

async function tryExample () {

  const pageFetcher = new TheCrawl(
    // Processor for page
    async (url, window) => {
      const { document } = window
      return {
        title: document.title,
        url,
      }
    }, {})
  const googleTopPage = await pageFetcher.process('https://www.google.com/')
  console.log('googleTopPage', googleTopPage)

}

tryExample().catch((err) => console.error(err))

API Guide

• the-crawl@1.0.0
  • create(args)
  • TheCrawl

License

This software is released under the MIT License.

Links

• THE Labo