Alis-request NPM

alis-request

Scraper api to get book by years on alis library

Installing

npm

npm install alis-request --save

yarn

yarn add alis-request

API

ReadableStreamBooks - read all book by year in stream
sendInitialQuery - make initial http query
getPage - get page body
getNumberedPageUrls - get ten page urls
run - recursive runner

Examples

You can do what you want with books, for example save in database. Next example return all books 2015 year in stream.

import Stream from 'stream';
import { sendInitialQuery, getPage, getNumberedPageUrls, run, ReadableStreamBooks } from 'alis-request';

const WritableStreamBooks = new Stream.Writable();
ReadableStreamBooks.pipe(WritableStreamBooks);

const books = [];

const initParams = {
  year: 2015,
  ip: '86.57.174.45',
};

sendInitialQuery(initParams, (err, res) => {
  if (err) {
    console.log(err);
    return;
  }
  const firstTenPageUrls = getNumberedPageUrls(res.page, initParams.ip);
  run(getPage, firstTenPageUrls, initParams.ip, res.jar);
});

WritableStreamBooks._write = (book, encoding, done) => {
  books.push(book);
  console.log(`STREAM: ${books.length}`);
  // ready to process the next chunk
  done();
};