db4bix v0.4.4

node-db4bix

Nodejs-rewritten DB monitoring plugin for Zabbix.

It is evolution of DBforBIX by SmartMarmot and DBforBIX by Vagabondan that were written in Java earlier.

Features

• Configuration is through Zabbix Web Interface
• Zabbix Preprocessing support!
• Whole table of items with only one SELECT: your databases remain in rest
• Connection pooling control: no reconnection DDoS from monitoring
• Several Zabbix Server instances support at a time
• Zabbix templates for every supported DB type
• Resolves Zabbix Host/Templates macros (i.e. {$DSN}, {$ANY_OTHER_STUFF}, etc.)
• Compatible with Zabbix 4.2.4++

Supported DB types

• MySQL
• PostgreSQL
• Oracle
• MSSQL

We are planning to extend it to other DBs...

You can easily extend this list, see ./libs/dbs/db-plugins

Documentation

• node-db4bix
  • Features
  • Supported DB types
  • Documentation
  • Installation and run
    • With Node.js
    • With docker-compose
    • With docker
  • Logs
    • How to read logs with docker-compose
    • How to read logs with docker
    • Choose log verbosity
  • Configuration
    • Configuration file db4bix.conf
    • Zabbix Server configuration items
      • XML syntax of Zabbix configuration items
  • How it works alltogether

Installation and run

With Node.js

1. Download this repo:

   git clone https://github.com/vagabondan/node-db4bix.git

2. Go to downloaded repo dir and install node packages:

   cd node-db4bix && npm install

3. Create appropriate db4bix.conf configuration file inside ./config subdirectory:

   cp ./config/db4bix_sample.conf ./config/db4bix.conf && vi ./config/db4bix.conf

4. Run:

   node index.js

   or with full debug messages: DEBUG='*' node index.js

With docker-compose

1. Create or download simple docker-compose.yml directly:

   wget https://raw.githubusercontent.com/vagabondan/node-db4bix/master/docker-compose.yml

   or just clone the whole repo and go inside created folder: git clone https://github.com/vagabondan/node-db4bix.git && cd node-db4bix

2. Create appropriate db4bix.conf configuration file inside ./config subdirectory, see example:

   mkdir ./config && vi ./config/db4bix.conf

3. Run:

   docker-compose up -d

You can choose appropriate debug level under environment section in docker-compose.yml by setting DEBUG variable. By default, it is set to info level.

With docker

1. Create appropriate db4bix.conf configuration file inside /path/to/config directory, see config file example:

   vi /path/to/config/db4bix.conf

2. Run docker:

   docker run -d -v /path/to/config:/app/config vagabondan/db4bix

   or with full DEBUG messages: docker run -d -v /path/to/config:/app/config -e 'DEBUG=*' vagabondan/db4bix

Logs

How to read logs with docker-compose

Read logs with command (from directory with docker-compose.yaml file):

docker-compose logs -f db4bix

How to read logs with docker

Do it with container id:

docker logs -f <container_id>

you can get container id with command: docker ps|grep db4bix this should read something like this: fd4c14dba6f6 vagabondan/db4bix "docker-entrypoint.s…" 5 seconds ago Up 3 seconds silly_mclean where the first hexademical identifier fd4c14dba6f6 is the container id.

Choose log verbosity

Log verbosity is managed with variable DEBUG. It is set differently depending of what type of db4bix run you have chosen:

1. Nodejs: right in the command line of terminal where you start db4bix:

   DEBUG='*' node index.js

2. docker-compose: inside docker-compose.yml file under environment section of db4bix service.
3. docker: as the parameter to docker run command right after -e option:

   docker run -d -v /path/to/config:/app/config -e 'DEBUG=*' vagabondan/db4bix

Below are some examples of DEBUG variable with short explanations:

1. Show all messages from all modules (not only db4bix):

   DEBUG='*'

2. Show debug messages from only db4bix modules:

   DEBUG='db4bix:*'

3. Show only info messages from only db4bix modules:

   DEBUG='db4bix:*:info'

4. Show all messages from ZabbixSender module of db4bix:

   DEBUG='db4bix:ZabbixSender:*'

5. Show only info messages from ZabbixSender module of db4bix:

   DEBUG='db4bix:ZabbixSender:info'

Configuration

DB4bix configuration is divided in two main parts described below:

1. Configuration file: db4bix.conf
2. Zabbix Server configuration items

Configuration file db4bix.conf

We use ini-file format and syntax inside db4bix.conf which is described in details here.

Configuration file keeps the following parameters listed in the table below:

Zabbix Server configuration items

Being connected to Zabbix Server DB4bix requests all configuration for its Zabbix Proxy name (it performs this action periodically according to parameter updateConfigPeriod defined at Global section of local DB4bix config file). It scans enabled hosts items for item keys ending with DB4bix.config (or what you have defined in configSuffix parameter under Zabbix section in local db4bix config file). You can specify any other config suffix in local DB4bix config at will, but important is to keep in mind that this suffix should uniquely identify items where DB4bix will search its configuration.

This Zabbix item keys (we will name it configuration item key) should also have 2 parameters and look like this:

.*DB4bix.config[someid,DBName]

where someid - some your identifier (whatever you want), DBName - DB name which should be present in local configuration file in dbs[] parameter under Zabbix section. It also may be a macro (we often use {$DSN} macro in our templates), then its value should be defined in host or template macros. Db4bix will resolve it prior to look for corresponding DB name at local config file.

Configuration items, i.e. those having item keys with .*DB4bix.config suffix should be of type Database monitor and contain DB4bix-like XML configuration in SQL query field, e.g.:

1. MySQL example where simple request returns result to single item key mysql.DB4bix.configversion,MySQL01:

   <parms type="mysql" prefix="mysql.">
     <server>
       <query time="600" item="DB4bix.config[version,MySQL01]">SHOW VARIABLES LIKE "version"</query>
     </server>
   </parms>

2. Another example for Oracle DB of Zabbix discovery item with item key:

   oracle.discovery.DB4bix.config[instanceid,{$DSN]

   It returns several results in one bulk request and has Zabbix host macro {$DSN} which value is defined at host macros with DB name of one of the databases defined at local DB4bix config file: select instid from gv$instance SELECT inst_id, REPLACE(name,' ',''), value FROM gv$sysstat WHERE name IN ('user I/O wait time','physical read total bytes','physical write total bytes','lob reads','lob writes','db block changes','db block gets','consistent gets','physical reads')

3. Many other examples you can find at Zabbix templates provided with this repo in ./templates subdirectory.

You can place in SQL field of DBforBix config item full XML config with root element - parms. You can make it as complex as you wish. DB4bix creates as many async code flows as needed to serve all config items and avoid mutual influence of different config items.

Final step you should do is to create items-receivers of data. We recommend to use type trappers for them but this is not necessary.

For second example above you should create item prototypes with keys, e.g.:

oracle.stats[{#INST_ID},user_I/O_wait_time] 
oracle.stats[{#INST_ID}physical_read_total_bytes] 
...
oracle.stats[{#INST_ID},physical_write_requests_optimized]

XML syntax of Zabbix configuration items

Two examples of XML syntax you've already seen earlier. Below is the table with all possible elements and attributes of XML configuration items.

<query time="60" item="one">select 1</query>

<multiquery time="60" item="anotherone">select 1</multiquery>

<query time="120" item="oneorzero" nodata="0">select 1</query>

<query time="60" item="one|two|three">select 1, 2, 3</query>

<query time="60" item="sameone|sametwo|samethree" type="column">select 1, 2, 3</query>

<query time="60" items="var1|var2" type="list">show variables</query>

<query time="60" items="variable%1" type="column">show variables</query>

</server></parms>

How it works alltogether

Local file config (db4bix.conf) defines which Zabbix Servers DB4bix may connect to. Also it defines how DB4bix may connect to databases which it can monitor.

DB4bix connects to Zabbix Server like an ordinary Zabbix Proxy. Of course for it may happen, corresponding Zabbix Proxy name should be defined at Zabbix Server and DB4bix should know this name too. This is the reason why you have to define proxyName parameter inside Zabbix.Name section of db4bix.conf.

DB4bix requests configuration data and Zabbix Server responds with information about all hosts attached to DB4bix proxy, items, templates, macros etc.

It is rather convenient and reasonable to create its own separate Zabbix Host to each DB instance, though nothing prevents you to keep configuration items of several DBs on one Zabbix Host. It is only our recommendations to separate them.

After configuration is received, DB4bix scans all items of all hosts for configSuffix and reads configurations from matching items. Items type should be Database monitor because only such item type has params field in its user web form that user can edit and read.

While reading configuration items DB4bix:

1. groups all queries by Database instances (second parameter in configuration item key) and time attributes (time interval between two consequent queries).
2. substitutes Zabbix macros with its values in item keys and queries
3. updates or creates periodical async monitors to query databases

Each monitor queries its database when time comes and saves results to local DB4bix storage mapping them to concrete Zabbix item ids.

Separate async periodic procedure sends all storage data with bulk request to Zabbix Servers according to sendDataPeriod parameters in Zabbix sections in local DB4bix config file.

Then everything repeats.