mongo-bench v0.4.1

MONGO-BENCH

Wrapper around mongodb instance which allows measuring execution time for mongo queries

FEATURES

This module provides the following:

• Logging query logs for all kinds of mongo queries (find, findOne, update, delete, findOneAndDelete, etc.).
• It works with cursors too (calculating total db work time when iterating through cursors)
• Subscribing to queries on database and collection levels using onQuery method
• Calculating query fingerprints (unless disabled). This helps finding similar queries. For example, following query:
  • { name: {$in: ["Joe", "Peter"] }, "age": { $gt: 20, $lt: 30 } } will have the same fingerprint as query
  • { age: { $lt: 50, $gt: 25 }, name: {$in: ["Ana", "Marry"] } }
• Support for the mongoose module. See how to use section

OPTIONS

There are currently only two options when using either MongoBench.mongo or MongoBench.db:

• logsCollection - specifies a collection where mongo will store logs. If not specified, logs will not be stored to mongo
• calculateQueryFingerprint - Unless this is set to false, query fingerprints will be calculated and added to each log. See more in fingeprint calculation

STATIC (LIBRARY) METHODS

This library provides a user two methods: MongoBench.mongo and MongoBench.db. First one can be used to set options (see section options) for the whole mongo library, while second option can be used to set database specific options. If database options are not set, they will inherit mongo options. It is not required to MongoBench.mongo, but then you'll at least have to MongoBench.db specific database instance using the following way: db = MongoBench.db(db, { ... opts ... }); Or to skip all that trouble, at the beginning of your program, just use: MongoBench.mongo( require('mongodb') ) Don't forget to check how to use section.

INSTANCE METHODS

Methods on both db and collection instances:

• onQuery - sets a listener for one of covered mongo operations. See examples in how to use section. If added on db instance level, it will capture all query actions on that database. If added on collection instance level, it will capture only actions for that collection.

Methods on db instance:

• setMongoBenchOptions - allows overriding MongoBench options set globally using MongoBench.mongo
• getOriginalInstance - gets original mongodb database connection instance

Methods on collection instance:

• skipNextLog - calling this method will skip logging for the next operation on that particular collection. You will (obviously) only be able to use this feature if you use native mongodb driver.

COVERED MONGO OPERATIONS

This module will capture mongo logs with the following mongo operations:

• count
• findOne
• findOneAndReplace
• findOneAndUpdate
• findOneAndDelete
• findAndRemove
• insert
• insertOne
• insertMany
• update
• updateOne
• updateMany
• delete
• deleteOne
• deleteMany
• remove
• find (returns cursor)
  • count
  • toArray
  • each
  • forEach
  • next (NOTE: triggers only when cursor reaches to the end returning total required mongo operation time)

WHAT'S IN EACH LOG?

Each log contains the following:

• action - mongo operation used (e.g. insertOne)
• duration - how long it took for mongo to complete the action (in miliseconds)
• timestamp - when did action occur (javascript Date object)
• database - which database was used for the action
• collection - collection this action was used on
• query - actual query that was used (in stringified form)
• queryFingerprint - fingerprint created from query (explained in "fingerprint calculation" section)

FINGERPRINT CALCULATION

To calculate query fingeprints, this app iterates through all sorted query keys. So there may be a lot of sorting, iteration, etc. If you don't need query fingerprints, it is suggested to turn them off by passing { calculateQueryFingeprint: false } as the second argument of MongoBench constructor

HOW TO USE

Below are few example code snippets which explain how to use this module.

WRITING QUERY LOGS BACK TO MONGODB

// first load mongodb and mongo-bench modules
const mongo = require('mongodb');
const MongoBench = require('mongo-bench');
// do the step below only once, in your main project file (first time you do a require('mongo') thing)
MongoBench.mongo(mongo, {
    'logsCollection': 'mongologs'  // logs will be written to this collection
});

// connect to mongo database
let server = new mongo.Server('10.42.1.31', 27017, {auto_reconnect: true, poolSize: 1});
let db = new mongo.Db(dbName, server, {native_parser: false, safe: true});
db.open();

db.collection('data').findOne({ id: 55 }).then(data => {
  // this query log is now written to 'mongologs' collection
});

SUBSCRIBING TO QUERY LOGS

// instead of being written to mongo, logs can be captured the following way
let col1 = db.collection('data1');
let col2 = db.collection('data2');

col1.onQuery(data => {
    // query logs only for collection1
});
db.onQuery(data => {
    // query logs for all *db* collection
});

col1.insertOne({ a: 1 });  // this will publish events both to col1 and db subscriber
col2.insertOne({ a: 2 });  // this will publish event only to db subscriber

SKIPPING LOGS

let col = db.collection('data');

col.onQuery(data => {
    // logs will be published here
});
col.skipLogging().findOne({ a: 1 });  // log for this insert action will not be emitted
col.findOne({ a: 1 });  // this query log will be emitted to that onQuery method up there

SKIPPING FINGERPRINT CALCULATION

// this example will show how you can skip fingerprint calculation
const mongo = require('mongodb');
const MongoBench = require('mongo-bench');
MongoBench.mongo(mongo, {
    'logsCollection': 'logs'
});

let db = new mongo.db( ... );
db.insertOne({ a: 1 });  // this will save query log to logs collection, without calculating query fingerprint

USING TWO DATABASES

// using more than one db connection? no problem
const mongo = require('mongodb');
const MongoBench = require('mongo-bench');
MongoBench.mongo(mongo);  // will not save logs to mongo, will calculate fingerprint

let db1 = new mongo.db( ... );
let db2 = new mongo.db( ... );
let col1 = db1.collection('col1');

col1.onQuery(data => {
    // this will, as usual capture only actions which happen on col1
});
db2.onQuery(data => {
    // and this here will only capture actions on db2
});

USING DIFFERENT SETUPS FOR DIFFERENT DATABASES

const mongo = require('mongodb');
const MongoBench = require('mongo-bench');
// first we set the defaults, which is calculating fingerprint only (turned on by default)
MongoBench.mongo(mongo);

// then we instantiate three different db connections
let db1 = new mongo.db( ... );
let db2 = new mongo.db( ... );
let db3 = new mongo.db( ... );

// and manually set parameters for each of them
db1.setMongoBenchOpitons({
    'logCollection': 'logs'  // this connection will save logs to mongo and calculate fingerprint
});
db1.setMongoBenchOpitons({
    'calculateQueryFingerprint': false  // this connection won't calculate fingerprint nor store mongo logs
});
// we don't touch db3, which means it will inhert its settings from MongoBench.mongo(mongo) setup

USING WITH MONGOOSE

// You can use this this library with mongoose along with one special requirement:
// require('mongoose') only after you do "MongoBench.mongo" stuff
// like in the example below (otherwise, it is not guaranteed to work)
const mongo = require('mongodb');
const MongoBench = require('mongo-bench');
MongoBench.mongo(mongo);
const mongoose = require('mongoose');

// do mongoose stuff like you usually do it
let SampleSchema = mongoose.schema({
    'name': String
});
let SampleModel = mongoose.model('col1', SampleSchema);

mongoose.connect('mongodb://localhost:27017/testdb').then(() => {
    // now do our magic stuff here
    let db = mongoose.connection.db;
    db.onQuery(data => {
        // once we call obj.save(), log will be available here
    });
    let obj = new SampleModel({ name: 'Joe' });
    obj.save();  // ta-da! That's it
});