logagent-output-clickhouse v1.0.0

Logagent Output ClickHouse Plugin

Output plugin for logagent to send logs to ClickHouse.

npm install -g logagent-output-clickhouse

Has the following improvements over the version bundled with logagent:

• if a field is blank or 0 then it would be set to null which ClickHouse would choke on
• can output different types of logs to different tables
• can include the database name in the table name to insert into different databases
• can ignore certain values (like "-" in empty nginx fields)
• buffers up the queries into batches before sending
• outputs geoip data into separate fields

Example configuration:

output:
  clickhouse:
    module: logagent-output-clickhouse
    url: http://127.0.0.1
    port: 8123
    # user: 'default'
    # password: ''
    # useGzip: true
    # database: db-name
    # debug: true
    batch: 15 # number of rows
    delay: 1000 # delay in ms before executing
    types:
      nginx:
        table: db.nginx
        # fields must be as per the order of the fields in the clickhouse table
        fields: [ts, status, method, host, path, size, duration, ip, ua, referer, geoip]
        ignore: ['-']
      nginx_err:
        table: db.nginx_err
        fields: [ts, level, message]