# datasift-push

> Node.js server for handling DataSift Push data over HTTP

Latest version **0.3.1** (published 2014-09-03) · MIT license · 0 weekly downloads

## Install

```sh
npm install datasift-push
pnpm add datasift-push
yarn add datasift-push
bun add datasift-push
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support; pre 1.0.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 0.3.1 |
| Published | 2014-09-03 |
| First published | 2013-10-10 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | CommonJS |
| Node | >=0.8 |
| Dependencies | 0 |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 1 |
| Author | Dirk Bonhomme |
| Maintainers | dirkbonhomme |
| Keywords | datasift, realtime, push, http, server |

## Links

- npm: https://www.npmjs.com/package/datasift-push
- Repository: https://github.com/dirkbonhomme/datasift-push
- Issues: https://github.com/dirkbonhomme/datasift-push/issues
- npm.io page: https://npm.io/package/datasift-push

## Alternatives

- [@opentelemetry/exporter-zipkin](https://npm.io/package/@opentelemetry/exporter-zipkin.md) — 14.8M weekly downloads
- [pusher-js](https://npm.io/package/pusher-js.md) — 2.0M weekly downloads
- [browserify](https://npm.io/package/browserify.md) — 1.7M weekly downloads
- [sqs-consumer](https://npm.io/package/sqs-consumer.md) — 1.7M weekly downloads
- [@sanity/eventsource](https://npm.io/package/@sanity/eventsource.md) — 930.8K weekly downloads

## Recent versions

- 0.3.1 (latest) — 2014-09-03
- 0.2.1 — 2013-12-06
- 0.2.0 — 2013-10-15
- 0.1.0 — 2013-10-10

## README

# DataSift Push Server for HTTP

This library is a Node.js implementation for DataSift's HTTP connector as described in their [HTTP docs](http://dev.datasift.com/docs/push/connectors/http). It should be used as a public endpoint for Push delivery.

## Usage overview

The following topics are covered:

* Installation
* Configuration
* Events
* FAQ
* Development & testing

## Installation

This library is not a stand-alone server but should be used as a dependency in an app that is able to handle incoming data.

Add `datasift-push` to your package.json dependencies and run the following command:

    $ npm install
    
Include it in your app:

    var DataSift = require('datasift-push');
    var server = new DataSift({ port: 8080 });
    server.on('data', datahandler);
    
## Configuration

#### `hostname` (String, optional)

Limit incoming requests to a certain hostname.

#### `sslKey` and `sslCert` (String or Buffer, optional)

Provide both your SSL private key and certificate to start an https server.

#### `port` (Number, optional)

Listen for incoming requests on a certain port. Defaults to 80.

#### `path` (String, optional)

Limit incoming requests to a certain path. Defaults to "/".  
A path of "/lorem/ipsum" will allow all requests to "/lorem/ipsum" and "/lorem/ipsum/dolor" but not "/lorem/sit/amet"

#### `format` (String, optional)

Format of incoming data. One of `json_meta`, `json_array` or `json_new_line`. Defaults to json_meta.  
Make sure that push subscriptions are created with the same format. The server will refuse all requests that cannot be parsed.

#### `maxSize` (Number, optional)

Maximum allowed size of request body (in bytes). Defaults to 20971520 (20MB, DataSift max)  
The server will immediately close the connection as soon as the size has been exceeded.

#### `confirmData` (Boolean, optional)

Expect app to confirm all emitted data. Defaults to false.  
When set to true, the server expects your app to confirm that it has processed incoming data. The `data` event will have a third parameter with a "done" callback. Call this to confirm that the incoming data has been saved. If your app does not respond within 10 seconds, the server will automatically respond with a HTTP 500 message. As a result, DataSift will try to resend the data after a certain interval.

#### `username` and `password` (String, optional)

Requires incoming requests to be authorized. Defaults to undefined.

### Example

    var server = new DataSift({
        port: 8080,
        path: '/lorem/ipsum/',
        username: 'john',
        password: 'loremipsum'
    });

## Events

#### `initialized`

Emitted when server starts listening on the provided port. Arguments: `port`, `hostname`

#### `refused`

Emitted when an incoming request has been refused. Arguments: `reason`

- Could not inflate compressed data
- Request did not contain required authentication headers
- Requested path "..." does not match "..."
- Post data exceeded limit of ... bytes
- Could not parse post data

#### `error` 

Emitted on internal errors. Arguments: `error` (Error instance)

#### `check`

Emitted after handling a "check" request from DataSift. Arguments: none

#### `data`

Emitted after parsing incoming data. Arguments: `data`, `meta`, `done` (optional)

- data: array of interactions and deletions
- meta: object with the request's meta data
    - url (raw url)
    - url_parsed (parsed url)
    - hash (playback id or stream hash)
    - hash_type (either historic or stream)
    - id (subscription id)
- done: function to be called after processing incoming data. Only provided when option `confirmData` is true. Call as `done(false)` to respond with a HTTP 500.

## FAQ

Q: I do not receive any incoming data  
A: Make sure the path, hostname and port are configured correctly and that your server is accessible from the internet.

Q: I keep receiving the same data  
A: Did you enable the `confirmData` option? Make sure you call the `done` function after processing incoming data. Make sure you are able to process incoming data in time.

Q: I am unable to see what's going on  
A: Listen to all described events to find out if there are any errors or refused connections. You might want to use `server.http` for more details debugging.


## Developing

The library is published to NPM and can be installed with the following command:

    $ npm install datasift-push

## Testing

Navigate to this module's repository and make sure you have the development modules installed:

    $ npm install


Run the tests:

    $ npm test

---
_Source: https://npm.io/package/datasift-push · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._