2.0.38 • Published 4 months ago

kafka-console v2.0.38

Weekly downloads
73
License
MIT
Repository
github
Last release
4 months ago

Kafka CLI tool

Command line tool to sufficiently and easy work with Kafka

NPM version Downloads

Table of Contents

Features

  • Producer
  • Consumer groups with seek and timeout
  • Built-in message encoders/decoders with types: json, js, raw
  • Custom message encoders/decoders as a js module
  • Message headers
  • GZIP compression
  • Plain, SSL and SASL_SSL implementations
  • Admin client
  • TypeScript support

Installing

npm install -g kafka-console

Examples

Common options

  -b, --brokers <brokers>                bootstrap server host (default: "localhost:9092")
  -l, --log-level <logLevel>             log level
  -t, --timeout <timeout>                set a timeout of operation (default: "0")
  -p, --pretty                           pretty print (default: false)
  --ssl                                  enable ssl (default: false)
  --mechanism <mechanism>                sasl mechanism
  --username <username>                  sasl username
  --password <password>                  sasl password
  --auth-id <authId>                     sasl aws authorization identity
  --access-key-id <accessKeyId>          sasl aws access key id
  --secret-access-key <secretAccessKey>  sasl aws secret access key
  --session-token <seccionToken>         sasl aws session token
  --oauth-bearer <oauthBearer>           sasl oauth bearer token
  -V, --version                          output the version number
  -h, --help                             display help for command

Commands

  consume [options] <topic>              Consume kafka topic events
  produce [options] <topic>              Produce kafka topic events
  metadata                               Displays kafka server metadata
  list|ls [options]                      Lists kafka topics
  config [options]                       Describes config for specific resource
  topic:create <topic>                   Creates kafka topic
  topic:delete <topic>                   Deletes kafka topic
  topic:offsets <topic> [timestamp]      Shows kafka topic offsets
  help [command]                         display help for command

Consumer

npx kafka-console consume [options] <topic>

Options

  -g, --group <group>              consumer group name (default: "kafka-console-consumer-TIMESTAMP")
  -d, --data-format <data-format>  messages data-format: json, js, raw (default: "json")
  -o, --output <filename>          write output to specified filename
  -f, --from <from>                read messages from the specific timestamp in milliseconds or ISO 8601 format. Set 0 to read from the beginning
  -c, --count <count>              a number of messages to read (default: null)
  -s, --skip <skip>                a number of messages to skip (default: 0)
  -h, --help                       display help for command

General usage with authentication

npx kafka-console --brokers $KAFKA_BROKERS --ssl --mechanism plain --username $KAFKA_USERNAME --password $KAFKA_PASSWORD consume $KAFKA_TOPIC --group $KAFKA_TOPIC_GROUP

Stdout from timestamp jq example

npx kafka-console consume $KAFKA_TOPIC --from 0 | jq .value

Custom data formatter example

npx kafka-console consume $KAFKA_TOPIC --data-format ./formatter/avro.js | jq

Producer

npx kafka-console produce [options] <topic>

Options

  -d, --data-format <data-format>  messages data-format: json, js, raw (default: "json")
  -i, --input <filename>           input filename
  -w, --wait <wait>                wait the time in ms after sending a message (default: 0)
  -h, --header <header>            set a static header (default: [])
  --help                           display help for command

General usage

npx kafka-console produce $KAFKA_TOPIC -b $KAFKA_BROKERS --ssl --mechanism plain --username $KAFKA_USERNAME --password $KAFKA_PASSWORD

Produce a json data from stdin with custom formatter

npx kafka-console payload.txt|kcli produce $KAFKA_TOPIC --data-format ./formatter/avro.js

Produce a json data from stdin

node payloadGenerator.js|npx kafka-console produce $KAFKA_TOPIC

Produce a json array data from stdin

cat payload.json|jq -r -c .[]|npx kafka-console produce $KAFKA_TOPIC

Payload single message input interface

interface Payload {
  key?: string; // kafka
  value: any;
  headers?: { [key: string]: value };
}

Formatters

export interface Encoder<T> {
  (value: T): Promise<string | Buffer> | string | Buffer;
}

export interface Decoder<T> {
  (value: Buffer): Promise<T> | T;
}

export interface Formatter<T> {
  encode: Encoder<T>;
  decode: Decoder<T>;
}

Supported Environment Variables

  • KAFKA_BROKERS
  • KAFKA_TIMEOUT
  • KAFKA_MECHANISM
  • KAFKA_USERNAME
  • KAFKA_PASSWORD
  • KAFKA_AUTH_ID
  • KAFKA_ACCESS_KEY_ID
  • KAFKA_SECRET_ACCESS_KEY
  • KAFKA_SESSION_TOKEN
  • KAFKA_OAUTH_BEARER

License

License The MIT License Copyright (c) 2024 Ivan Zakharchanka

2.0.19

8 months ago

2.0.18

8 months ago

2.0.26

6 months ago

2.0.27

6 months ago

2.0.24

7 months ago

2.0.25

7 months ago

2.0.22

7 months ago

2.0.23

7 months ago

2.0.20

8 months ago

2.0.21

7 months ago

2.0.28

6 months ago

2.0.29

6 months ago

2.0.37

4 months ago

2.0.38

4 months ago

2.0.35

4 months ago

2.0.36

4 months ago

2.0.33

5 months ago

2.0.34

4 months ago

2.0.31

5 months ago

2.0.32

5 months ago

2.0.30

6 months ago

2.0.17

8 months ago

2.0.15

8 months ago

2.0.16

8 months ago

2.0.13

9 months ago

2.0.14

9 months ago

2.0.11

9 months ago

2.0.12

9 months ago

2.0.10

9 months ago

2.0.3

10 months ago

2.0.2

10 months ago

2.0.5

10 months ago

2.0.4

10 months ago

2.0.7

9 months ago

2.0.6

9 months ago

2.0.9

9 months ago

2.0.8

9 months ago

2.0.1

10 months ago

2.0.0

10 months ago

1.4.48

10 months ago

1.4.49

10 months ago

1.4.50

10 months ago

1.4.28

1 year ago

1.4.27

1 year ago

1.4.29

1 year ago

1.4.31

1 year ago

1.4.30

1 year ago

1.4.33

1 year ago

1.4.32

1 year ago

1.4.35

1 year ago

1.4.34

1 year ago

1.4.37

1 year ago

1.4.36

1 year ago

1.4.39

1 year ago

1.4.38

1 year ago

1.4.40

1 year ago

1.4.42

11 months ago

1.4.41

1 year ago

1.4.44

11 months ago

1.4.43

11 months ago

1.4.46

11 months ago

1.4.45

11 months ago

1.4.47

11 months ago

1.4.26

1 year ago

1.4.25

1 year ago

1.4.24

1 year ago

1.4.23

1 year ago

1.4.22

1 year ago

1.4.20

1 year ago

1.4.21

1 year ago

1.4.19

1 year ago

1.4.18

1 year ago

1.4.17

1 year ago

1.4.16

1 year ago

1.4.15

1 year ago

1.4.14

1 year ago

1.4.13

1 year ago

1.4.12

1 year ago

1.4.11

1 year ago

1.4.10

1 year ago

1.4.9

1 year ago

1.4.8

1 year ago

1.4.7

1 year ago

1.4.6

1 year ago

1.4.5

1 year ago

1.4.4

1 year ago

1.4.3

1 year ago

1.4.2

1 year ago

1.4.1

1 year ago

1.4.0

1 year ago

1.3.53

1 year ago

1.3.52

1 year ago

1.3.51

1 year ago

1.3.50

1 year ago

1.3.49

1 year ago

1.3.48

1 year ago

1.3.47

1 year ago

1.3.46

1 year ago

1.3.45

1 year ago

1.3.44

1 year ago

1.3.43

1 year ago

1.3.42

2 years ago

1.3.41

2 years ago

1.3.39

2 years ago

1.3.37

2 years ago

1.3.38

2 years ago

1.3.40

2 years ago

1.3.36

2 years ago

1.3.32

2 years ago

1.3.35

2 years ago

1.3.33

2 years ago

1.3.34

2 years ago

1.3.31

2 years ago

1.3.30

2 years ago

1.3.29

2 years ago

1.3.17

2 years ago

1.3.18

2 years ago

1.3.16

2 years ago

1.3.19

2 years ago

1.3.20

2 years ago

1.3.21

2 years ago

1.3.24

2 years ago

1.3.25

2 years ago

1.3.22

2 years ago

1.3.23

2 years ago

1.3.28

2 years ago

1.3.26

2 years ago

1.3.27

2 years ago

1.3.9

2 years ago

1.3.10

2 years ago

1.3.13

2 years ago

1.3.14

2 years ago

1.3.11

2 years ago

1.3.12

2 years ago

1.3.15

2 years ago

1.3.8

2 years ago

1.2.61

2 years ago

1.3.7

2 years ago

1.3.6

2 years ago

1.3.5

2 years ago

1.3.4

2 years ago

1.3.3

2 years ago

1.3.2

2 years ago

1.3.1

2 years ago

1.3.0

2 years ago

1.2.60

3 years ago

1.2.59

3 years ago

1.2.56

3 years ago

1.2.57

3 years ago

1.2.58

3 years ago

1.2.41

3 years ago

1.2.42

3 years ago

1.2.45

3 years ago

1.2.46

3 years ago

1.2.43

3 years ago

1.2.44

3 years ago

1.2.49

3 years ago

1.2.47

3 years ago

1.2.48

3 years ago

1.2.52

3 years ago

1.2.53

3 years ago

1.2.50

3 years ago

1.2.51

3 years ago

1.2.54

3 years ago

1.2.55

3 years ago

1.2.40

3 years ago

1.2.31

3 years ago

1.2.34

3 years ago

1.2.35

3 years ago

1.2.32

3 years ago

1.2.33

3 years ago

1.2.38

3 years ago

1.2.39

3 years ago

1.2.36

3 years ago

1.2.37

3 years ago

1.2.27

3 years ago

1.2.28

3 years ago

1.2.26

3 years ago

1.2.29

3 years ago

1.2.30

3 years ago

1.2.24

3 years ago

1.2.25

3 years ago

1.2.0

4 years ago

1.2.8

3 years ago

1.2.7

3 years ago

1.2.6

4 years ago

1.2.5

4 years ago

1.2.4

4 years ago

1.2.3

4 years ago

1.2.2

4 years ago

1.2.1

4 years ago

1.2.12

3 years ago

1.2.13

3 years ago

1.2.10

3 years ago

1.2.11

3 years ago

1.2.16

3 years ago

1.2.17

3 years ago

1.2.14

3 years ago

1.2.15

3 years ago

1.2.18

3 years ago

1.2.19

3 years ago

1.1.9

4 years ago

1.1.8

4 years ago

1.2.20

3 years ago

1.2.23

3 years ago

1.1.12

4 years ago

1.1.11

4 years ago

1.2.21

3 years ago

1.1.10

4 years ago

1.2.22

3 years ago

1.1.16

4 years ago

1.1.15

4 years ago

1.1.14

4 years ago

1.1.13

4 years ago

1.2.9

3 years ago

1.1.7

4 years ago

1.1.6

4 years ago

1.1.5

4 years ago

1.1.4

4 years ago

1.1.3

4 years ago

1.1.1

4 years ago

1.1.2

4 years ago

1.1.0

4 years ago

1.0.7

4 years ago

1.0.5

4 years ago

1.0.4

4 years ago

1.0.3

4 years ago

1.0.2

5 years ago

1.0.1

5 years ago

1.0.0

5 years ago

0.3.7

5 years ago

0.3.6

5 years ago

0.3.5

5 years ago

0.3.4

5 years ago

0.3.3

5 years ago

0.3.2

5 years ago

0.3.0

5 years ago

0.3.1

5 years ago

0.2.0

5 years ago

0.1.1

5 years ago

0.1.0

5 years ago