@inspr/kafka-promise v1.0.4

Simple kafka promise

This library is based on node-rdkafka and aim to provide a simple producer and consumer wrapped with promise to allow the use of async / await with minimal configuration and overhead.

This library is fully written in TypeScript.

Latest release 2.0.0 - 2020-01-14

Added

• (consumer) added consumer interface

Changed

• (index) changed export style to typescript
• (producer) migration of high level producer to typescript
• (consumer) migration of kafka consumer to typescript
• (mock) migration of mocked consumer to typescript
• (mock) migration of mocked producer to typescript
• (examples) updated examples for producer and consumer

Fixed

• (producer) fixed thrown error if kafka.producer.topicsPrefix was missing

Removed

• (producer) removed kafkaNProducer
• (producer) removed poll() before disconnecting
• (dependencies) removed unused hirestime and uuid dependency
• (dependencies) removed unused prom-client, s3pweb-logger and config dependency

Breaking changes from 1.x.x

• Producer and consumer now are classes and have a constructor
• No more direct need for node-config and a logger

Installation

npm install @inspr-npm/kafka-promise

Configuration

Minimal configuration for the consumer is:

{ "metadata.broker.list": "0.0.0.0:9094", "group.id": "test.group" }

Minimal configuration for the producer is:

{ "metadata.broker.list": "0.0.0.0:9094" }

This project is based on node-rdkafka and supports the same configuration options. Go here for more details.

Create a consumer instance

const KafkaConsumer = require('@inspr-npm/kafka-promise').KafkaConsumer

// Create a new instance
const consumer = new KafkaConsumer({ 'metadata.broker.list': '0.0.0.0:9094', 'group.id': 'test.group' }, 1000)

// Connect
await consumer.connect(['topicName'])

// Consume messages
const messagesArray = await consumer.listen(100, true)

// Disconnect the consumer
await consumer.disconnect()

To use with typescript, just change the import to

import { KafkaConsumer } from '@inspr-npm/kafka-promise';

Create a producer instance

const KafkaProducer = require('@inspr-npm/kafka-promise').KafkaProducer

// Create a new instance
const producer = new KafkaProducer({ 'metadata.broker.list': '0.0.0.0:9094' }, '')

// Connect
await producer.connect(['topicName'])

// Produce some messages
const offset = await producer.sendMessage(topicName, { message: `My message.` }, 0, null)

// Disconnect
await producer.disconnect()

To use with typescript, just change the import to

import { KafkaProducer } from '@inspr-npm/kafka-promise';

Example

To produce some messages take a look at ./examples/producer.js and to consume some messages take a look at ./examples/consumer.js.

If you have docker, you can use ./examples/docker-compose.yaml to start one zookeeper and one kafka stack on your machine. This stack comes with kafkamanager and kafkadrop for easy monitoring and debugging.

Docker

If you want to build a docker image based on alpine linux, you need to add some packages to the base image. Go here for more details.