0.0.27 • Published 2 years ago

@compass-marketing/puppet-stream v0.0.27

Weekly downloads
18
License
ISC
Repository
github
Last release
2 years ago

Puppet Stream

Puppet Stream is a node library that makes it easy to stream scraping requests to Puppeteer using native NodeJS streams.

Getting Started

Install

  npm i @compass-marketing/puppet-stream

Important: Nightcrawler has a dependency to puppeteer-core. This allows for a custom executable to be used if desired. If you would like to use the chrome executable bundled with puppeteer, please install it like so:

npm i puppeteer

Usage

const Nightcrawler = require('@compass-marketing/puppet-stream')
const JSONStream = require('JSONStream')

let nc = new Nightcrawler()
let qStream = nc.createStream()

// Pipe to standard out
qStream.pipe(JSONStream.stringify(false)).pipe(process.stdout)

let q = nc
  .get('http://example.com')
  .waitFor('body')
  .groupBy('body > div')
  .select({ title: nc.$('p') })

qStream.write(q)

qStream.end()
0.0.27

2 years ago

0.0.24

2 years ago

0.0.25

2 years ago

0.0.26

2 years ago

0.0.23

4 years ago

0.0.21

4 years ago

0.0.22

4 years ago

0.0.20

4 years ago

0.0.19

5 years ago

0.0.18

5 years ago

0.0.17

5 years ago

0.0.16

5 years ago

0.0.15

6 years ago