terafoundation_kafka_connector v0.12.0

Kafka Connector

Terafoundation connector for Kafka producer and consumer clients.

To install from the root of your terafoundation based service.

npm install terafoundation_kafka_connector

Configuration:

The terafoundation level configuration is as follows:

When using this connector in code, this connector exposes two different client implementations. One for producers type: producer and one for consumers type: consumer.

The options object enables setting a few properties

Consumer connector configuration example:

{
    options: {
        type: 'consumer',
        group: 'example-group'
    },
    topic_options: {
        'enable.auto.commit': false
    },
    rdkafka_options: {
        'fetch.min.bytes': 100000
    }
}

Producer connector configuration example:

{
    options: {
        type: 'producer',
        poll_interval: 1000,
    },
    topic_options: {},
    rdkafka_options: {
        'compression.codec': 'gzip',
        'topic.metadata.refresh.interval.ms': -1,
        'log.connection.close': false,
    }
}

Terafoundation configuration example:

terafoundation:
    connectors:
        kafka:
            default:
                brokers: "localhost:9092"