# paged-http-stream

> Turn a paged http request into a stream of pages.

Latest version **1.0.1** (published 2015-09-27) · BSD license · 0 weekly downloads

## Install

```sh
npm install paged-http-stream
pnpm add paged-http-stream
yarn add paged-http-stream
bun add paged-http-stream
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 1.0.1 |
| Published | 2015-09-27 |
| First published | 2015-09-25 |
| Weekly downloads | 0 |
| License | BSD |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 4 |
| Known vulnerabilities | 0 (+1 in 1 direct dependencies) |
| Install scripts | no |
| Author | Karissa McKelvey |
| Maintainers | karissa |

## Links

- npm: https://www.npmjs.com/package/paged-http-stream
- npm.io page: https://npm.io/package/paged-http-stream

## Dependencies (4)

- [got](https://npm.io/package/got.md) ^4.2.0
- [debug](https://npm.io/package/debug.md) ^2.2.0
- [event-stream](https://npm.io/package/event-stream.md) ^3.3.1
- [concat-stream](https://npm.io/package/concat-stream.md) ^1.5.0

## Recent versions

- 1.0.1 (latest) — 2015-09-27
- 1.0.0 — 2015-09-25

## README

# paged-http-stream

Turn a paged http request into a stream of pages. Focuses on simplicity and modularity. Doesn't try to assume anything. You must implement a 'next' function that returns the next request based upon the previous one.

```
npm install paged-http-stream
```

## API

### `var pages = pager(opts, next)`

Options:

`method`: defaults to GET

`uri`: the url to query. includes query string.

... and anything else that can be passed to a typical node.js http request (uses [got](http://npmjs.org/got))

### `next = function (data)`

You need to implement the `next` function, which be able to interpret the data from the previous request, passed in as an argument, into a new request.

## Example

For example, [figshare](http://figshare.com) returns each of their search pages like this:
```json
{
  "items_found" : 91,
  "page_nr": 1,
  "items": []
}
```

So, I can write a function `next` that takes the `page_nr` and adds one, and then returns `null` when the items list is empty:
```js
function next (data) {
  // data will be JSON parsed already.
  if (data.error) throw new Error(data.error)
  if (data.items && data.items.length === 0) return null // we are done here
  var query = {
    search_for: 'this is my query',
    page: parseInt(data.page_nr + 1) || 1 // get the next page
  }
  return getOpts(query)
}

function getOpts (query) {
  return {
    method: 'GET',
    uri: 'http://api.figshare.com/v1/articles/search?' + qs.stringify(query)
  }
}
```

Then, pass to the pager:
```js
var pageStream = pager(startingOpts, next)
```

You can then get access to each page like this (as JSON):
```js
pageStream.on('data', funciton (page) {
  console.log(page.items)
})
```

## As seen in

[figshare-search](http://github.com/karissa/figshare-search)

## TODO

* timeout, or wait time between requests

---
_Source: https://npm.io/package/paged-http-stream · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._