Inote-util NPM

inote-util

A collection of utility functions and classes for Node.js.

Features - method-by-method description of the utilities.
Installing - how to install from npm or source.
Licensing - inote-util is made available under an MIT License.
How to Contribute - tips on reporting issues, creating pull-requests for or just generally hacking inote-util.

Using

note-util exports a number of objects, each containing several utility functions.

Hence, require(‘note-util’) will return a map with several sub-objects (ArrayUtil, AsyncUtil, etc.)

There are two common idioms for handling this.

Idiom 1: Keep a handle to this “parent” object:

var IU = require(“note-util”);

IU.LogUtil.tlog(“Here is a random string:”, IU.RandomUtil.random_alphanumeric());

Idiom 2: Keep a handle to the individual utilities:

var LogUtil = require(“note-util”).LogUtil;
var RandomUtil = require(“note-util”).RandomUtil;

LogUtil.tlog(“Here is a random string:”, RandomUtil.random_alphanumeric());

Personally, the author prefers the second approach, but the difference is almost entirely cosmetic. In particular, note that in Node.js require-ing the same module multiple times DOES NOT reload or reparse the files in the module. The file is only load once.

The following section lists methods under the associated object. E.g., under NetUtil you’ll find get_unused_port(). This means that the get_unused_port method is exported within the top-level object named “NetUtil” and could be accessed via:

var gup = require(“note-util”).NetUtil.get_unused_port;

Features

Feature Index

ArrayUtil

lpad(value,width,pad) - adds pad elements to the beginning of value until value is width elements long. (Also accepts strings, see StringUtil.lpad, which is identical.)
rpad(value,width,pad) - adds pad elements to the end of value until value is width elements long. (Also accepts strings, see StringUtil.rpad, which is identical.)
smart_join(array,delim,last_delim) - identical to Array.join, except the specified last_delim is used between the last two elements (if any). E.g., smart_join(["Tom","Dick","Harry"],", "," and ") yields Tom, Dick and Harry.
trim_trailing_null(array) - returns a copy of array with trailing null elements removed
right_shift_args(...) - returns an array the same length as the given arguments, but any trailing null values are converted to leading null values. (Most useful in the CoffeeScript idiom [a,b,c,d] = right_shift_args(a,b,c,d).)
paginate_list(list[,offset=0,limit=20]) - returns the specified section of the given array.
subset_of(a,b) / ArrayUtil.is_subset_of(a,b) - returns true if every element of array a is also an element of b.
strict_subset_of(a,b) / ArrayUtil.is_strict_subset_of(a,b) - returns true if every element of array a appears exacty the same number of times in array b as it does in array a. (E.g., ['a','a','b'] is subset of but not a strict subset of ['a','b','c'], according to this definition).
sets_are_equal(a,b) - compares arrays as if they were sets.
arrays_are_equal(a,b) - true if and only if array a and array b contain the exact same elements in the exact same order. DEPRECATED: please use ObjectUtil.deep_equal
uniquify(array,key) - returns a clone of array with duplicate values removed. When the array contains objects (maps) and a key is provided, two elements will be considered duplicates if they have the same value for the attribute key.

Back to Index

AsyncUtil

wait_until(predicate,interval,callback) / wait_for(predicate,interval,callback) - polls predicate() every interval milliseconds until it returns true (or throws an exception). Callback signature is (err,complete) where complete is true if and only if predicate() returned true. interval defaults to 100.
wait(delay,callback) / set_timeout(delay,callback) / setTimeout(delay,callback) - just like setTimeout(callback,delay) but with a more CoffeeScript-friendly parameter order.
cancel_wait(id) / clear_timeout(id) / clearTimeout(id) - alias for window.clearTimeout(id).
interval(delay,callback) / set_interval(delay,callback) / setInterval(delay,callback) - just like setInterval(callback,delay) but with a more CoffeeScript-friendly parameter order.
cancel_interval(id) / cancelInterval(id) / clear_interval(id) / clearlInterval(id) - alias for window.clearInterval(id).
for_async(initialize,condition,action,increment,options,whendone) - executes an asynchronous for loop. Accepts 5 function-valued parameters:
- initialize - an initialization function (no arguments passed, no return value is expected);
- condition - a predicate that indicates whether we should continue looping (no arguments passed, a boolean value is expected to be returned);
- action - the action to take (a single callback function is passed and should be invoked at the end of the action, no return value is expected);
- increment - called at the end of every action, prior to condition (no arguments passed, no return value is expected);
- options - optional options map
  - delay - time added before each successive invocation, in milliseconds
  - timeout - max time to wait for an action to complete, in milliseconds
  - catch_exceptions - boolean
- whendone - called at the end of the loop (when condition returns false), (no arguments passed, no return value is expected).
for_each_async(list,action,options,whendone) - executes an asynchronous forEach loop. Accepts 3 (or 4) parameters:
- list - the array to iterate over;
- action - a function with the signature (value,index,list,next) indicating the action to take for each element (must call next for processing to continue);
- options - optional options map
  - delay - time added before each successive invocation, in milliseconds
  - timeout - max time to wait for an action to complete, in milliseconds
  - catch_exceptions - boolean
- whendone - called at the end of the loop.
fork(methods,args_for_methods,options,callback) - runs the given array of methods "simultaneously" (asynchronously), invoking callback when they are all complete:
- options - optional options map
  - delay - time added before each successive invocation, in milliseconds
  - timeout - max time to wait for an action to complete, in milliseconds
  - catch_exceptions - boolean
throttled_fork(max_parallel,methods,args_for_methods,options,callback) - just like fork, but never running more than max_parallel functions at the same time.
fork_for_each_async(list,action,options,whendone) - like for_each_async but running action in parallel for each element of the list. The whendone callback is provided with a list of "responses", in the same order as the original list.
throttled_fork_for_each_async(max_parallel,list,action,options,whendone) - like fork_for_each_async but running at most max_parallel methods at any one time.
invoke_with_timeout(method,args,options,callback)
maybe_invoke_with_timeout(method,args,options,callback)
procedure() - generates a Sequencer object, as described below

The Sequencer

The methods Util.procedure(), AsyncUtil.procedure(), and new Sequencer() will create a new "sequencer" object that provides a simple way to "flatten" long chains of nested asynchronous methods.

For example, rather than writing:

method_one( function(e,a,b) {
  method_two(a, b, function() {
    method_three( function(e,c,d) {
      and_so_on();
    });
  });
});

We can flatten the calls out like this:

var procedure = AsyncUtil.procedure();
procedure.first( method_one );
procedure.then( method_two );
procedure.then( method_three );
procedure.then( and_so_on );
procedure.run();

Each call to then appends the given callback method to the chain.

Each callback method is passed a next_step function that must be called to trigger the next step in the processing.

Note that any arguments passed to next_step() will be passed to the next method in the sequence (with the next function appended). For example, given a method in the sequence such as:

procedure.next(function(next_step) {
  next_step(1,"A",[]);
});

The following method in the procedure will be invoked with the following signature:

the_next_method_in_the_sequence(1,"A",[],next_step);

Hence the typical use of the class looks something like this:

var s = new Sequencer()
s.first( function(done) {
  // do something, then invoke the callback
  done();
});

s.next( function(done) {
  // do something, then invoke the callback
  done();
});
s.next( function(done) {
  // do something, then invoke the callback
  done();
});

s.run();

When run is invoked, each asynchronous method is executed in sequence.

The first method is optional (you can just use next instead), but when invoked first will remove any methods previously added to the chain.

You last methods is an optional short-hand for adding one final method to the chain and then running it. E.g., the last two lines of our example:

procedure.then( and_so_on )
procedure.run()

Could be re-written:

procedure.last( and_so_on )

Note that the sequence is not cleared when run is invoked, so one may invoke run more than once to execute the sequence again.

Back to Index

ColorUtil

hex_to_rgb_triplet(hex) - converts a hex-based #rrggbb string to decimal [r,g,b] values.
hex_to_rgb_string(hex) - converts a hex-based #rrggbb string to a string of the form rgb(r,g,b).
rgb_string_to_triplet(rgb) - converts a string of the form rgb(r,g,b) to decimal [r,g,b] values.
rgb_triplet_to_string(r,g,b) - convert an array or sequence of r, g, b values to a string of the form rgb(r,g,b).

Back to Index

Config

A thin wrapper around nconf providing a consistent way to load configuration data from files or the environment.

Note that config now allows // and /* */ style comments in JSON files.

Example of use

var config = require('inote-util').config.init();
var prop = config.get('my:property');

Config will load the configuration from several sources.

In order of precedence:

"Override" values passed to the init function.
Command line parameters (--name value).
Environment variables. Note that __ can be used in place of : as a delimiter.
A JSON-format configuration file (from a location deterimined by NODE_ENV, config_dir or config_file).
A "default" JSON-format configuration file at ${config_dir}/config.json.
"Default" values passed to the init function.

To discover a configuration file (as used in step 3 above), Config will:

Look for NODE_ENV, config_dir or config_file environment variables.
If config_dir is set, the value will be used as the "parent" directory of the configuration files. (If config_dir is not set, it defaults to the directory config within the working directory from which the current Node.js process was launched.)
If NODE_ENV is set and a file named ${NODE_ENV}.json exists within the config_dir, that file will be used.
If config_file is set, that file will be used.

Back to Index

DateUtil

start_time - timestamp at which inote-util was loaded (hence approximately the time the application was started in most circumstances).
duration(end_time,begin_time) - returns an object that breaks-down the time between begin_time and end_time in several ways (as described below). When missing, end_time defaults to Date.now() and begin_time defaults to start_time.
iso_8601_regexp() - returns a regular expression that can be used to validate an ISO 8601 formatted date.

Here is an example of the object returned by the DateUtil.duration, with brief annotations.

{
  "begin":1462806430444,
  "end":1462851730757,
  "delta":45300313,
  "in_millis":{                     // MILLISECOND VALUE OF EACH "PART" OF THE DURATION
    "millis":313,                   // <= delta % (1000)
    "seconds":313,                // <= delta % (1000 * 60)
    "minutes":2100313,              // <= delta % (1000 * 60 * 60)
    "hours":45300313,               // <= delta % (1000 * 60 * 60 * 24)
    "days":45300313,                // <= delta % (1000 * 60 * 60 * 24 * 7
    "weeks":45300313,               // <= delta % (1000 * 60 * 60 * 24 * 7 * 52
    "years":45300313                // <= delta
  },
  "raw":{                           // ELEMENTS FROM `IN_MILLIS`, CONVERTED TO RELEVANT UNIT
    "millis":313,                   // <= in_millis.millis
    "seconds":0.313,                // <= in_millis.seconds / (1000)
    "minutes":35.00521666666667,    // <= in_millis.minutes / (1000 * 60)
    "hours":12.583420277777778,     // <= in_millis.hours   / (1000 * 60 * 60)
    "days":0.5243091782407407,      // <= in_millis.days    / (1000 * 60 * 60 * 24)
    "weeks":0.07490131117724867,    // <= in_millis.weeks   / (1000 * 60 * 60 * 24 * 7)
    "years":0.00020520907171848953   // <= in_millis.years   / (1000 * 60 * 60 * 24 * 7 * 365)
  },
  "whole":{                         // RAW VALUES ROUNDED DOWN TO NEAREST INTEGER
    "millis":313,
    "seconds":0,
    "minutes":35,
    "hours":12,
    "days":0,
    "weeks":0,
    "years":0
  },
  "array":{                         // SET OF DURATION ELEMENTS IN ARRAYS
    "full":{                        // FULL = ALL UNITS, EVEN WHEN 0
      "values":[0,0,0,12,35,0,313],
      "short":["0y","0w","0d","12h","35m","0s","313m"],
      "long":["0 years","0 weeks","0 days","12 hours","35 minutes","0 seconds","313 milliseconds"],
      "no_millis":{                 // SAME AS PARENT BUT IGNORING MILLISECONDS
        "values":[0,0,0,12,35,0],
        "short":["0y","0w","0d","12h","35m","0s"],
        "long":["0 years","0 weeks","0 days","12 hours","35 minutes","0 seconds"]
      }
    },
    "brief":{                       // BRIEF = SKIP TO LARGEST NON-ZERO UNIT, THEN INCLUDE ALL
      "values":[12,35,0,313],
      "short":["12h","35m","0s","313m"],
      "long":["12 hours","35 minutes","0 seconds","313 millis"],
      "no_millis":{
        "values":[12,35,0],
        "short":["12h","35m","0s"],
        "long":["12 hours","35 minutes","0 seconds"]
      }
    },
    "min":{                        // MIN = ONLY THE NON-ZERO VALUES

      "units":["hour","minute","millisecond"],
      "short":["12h","35m","313m"],
      "long":["12 hours","35 minutes","313 milliseconds"],
      "no_millis":{
        "units":["hour","minute"],
        "short":["12h","35m"],
        "long":["12 hours","35 minutes"]
      }
    }
  },
  "string":{                       // SIMILAR TO "ARRAY" BUT WITH STRINGS
    "full":{
      "micro":"0y0w0d12h35m0s313m",
      "short":"0y 0w 0d 12h 35m 0s 313m",
      "long":"0 years 0 weeks 0 days 12 hours 35 minutes 0 seconds 313 milliseconds",
      "verbose":"0 years, 0 weeks, 0 days, 12 hours, 35 minutes, 0 seconds and 313 milliseconds",
      "no_millis":{
        "micro":"0y0w0d12h35m0s",
        "short":"0y 0w 0d 12h 35m 0s",
        "long":"0 years 0 weeks 0 days 12 hours 35 minutes 0 seconds",
        "verbose":"0 years, 0 weeks, 0 days, 12 hours, 35 minutes and 0 seconds"
      }
    },
    "brief":{
      "micro":"12h35m0s313m",
      "short":"12h 35m 0s 313m",
      "long":"12 hours 35 minutes 0 seconds 313 millis",
      "verbose":"12 hours, 35 minutes, 0 seconds and 313 millis",
      "no_millis":{
        "micro":"12h35m0s",
        "short":"12h 35m 0s",
        "long":"12 hours 35 minutes 0 seconds",
        "verbose":"12 hours, 35 minutes and 0 seconds"
      }
    },
    "min":{
      "micro":"12h35m313m",
      "short":"12h 35m 313m",
      "long":"12 hours 35 minutes 313 milliseconds",
      "verbose":"12 hours, 35 minutes and 313 milliseconds",
      "no_millis":{
        "micro":"12h35m",
        "short":"12h 35m",
        "long":"12 hours 35 minutes",
        "verbose":"12 hours, 35 minutes and 48 seconds"
      }
    }
  }
}

Back to Index

FileUtil

file_age(file,callback) / file_age_sync(file) - obtain the age of a file (time since last modfied) in milliseconds
file_mtime(file,callback) / file_mtime_sync(file)- obtain the Unix epoch timestamp at which a file was last modified
ls(dir[,options],callback) - list the files in a directory; options:
- recurse - when true, perform the operation recursively
- pattern - when a non-null RegExp, only list files matching the specified pattern
- types - an array or string containing file or directory
is_dir(filename,callback) - test if the specified filename is a directory
is_file(filename,callback) - test if the specified filename is a plain file (not a directory).
sanitize_filename(filename) - removes invalid characters from and truncates extremely long filenames; only operates on the last segement of the given path. That is, if filename is /foo/bar/xyz, only the xyz part of the string will be modified.
uniquify_filename(dir,basename[,ext=''[,minpadwidth=3,maxpadwidth=5]) - attempts to generate a unique filename in dir based on basename.
mkdir(dir) - mkdir -p dir
touch(file) - touch file
rm(files...) - remove one or more files, ignoring errors. (Returns true if any errors are encountered, false otherwise).
rmdir(dirs...) - recursively remove one or more directories or files, ignoring errors. (Returns true if any errors are encountered, false otherwise).
read_stdin_sync([end_byte="\x04",buffer_size=512]) - synchronously read all of stdin (up to end_byte), returning the resulting buffer
load_json_file(file, options, callback) (also just load_json)- asynchronously read and parse a JSON file. When options.ignore_errors is true, calls-back with null, null rather than err, null. When options.allow_comments is true (the default) JS-style comments are allowed. When options.strip_comments is true (the default) comments do NOT appear in the returned JSON object.
load_json_file_sync(file,options) - synchronously read and parse a JSON file. When options.ignore_errors is true, returns null rather than throwing an exception when the file is not found or does not contain value JSON. When options.allow_comments is true (the default) JS-style comments are allowed. When options.strip_comments is true (the default) comments do NOT appear in the returned JSON object.
load_json_stdin_sync([end_byte="\x04"[,buffer_size=512,options]]) - synchronously read and parse JSON object from stdin. When options.ignore_errors is true, returns null rather than throwing an exception. When options.allow_comments is true (the default) JS-style comments are allowed. When options.strip_comments is true (the default) comments do NOT appear in the returned JSON object.
copy_file(src,dest,callback) - copy a file from src to dest; works across filesystems.
move_file(src,dest,callback) - move (rename) a file from src to dest; works across filesystems.

Back to Index

FileUtil - MIME and File-Extension related

get_ext(fname) / get_extension(fname) - equivalent to path.extname, save that fname can be the extension itself. (E.g., both path.extname('.foo') and path.extname('foo') return '' while FileUtil.get_ext('.foo') and FileUtil.get_ext('foo') return 'foo'). Specifically, returns the part of fname following the final dot (.), unless fname contains no dot, in which case the entire fname value is returned.
strip_ext(fname) / strip_extension(fname) - returns a version of fname with the file extension removed.
replace_ext(fname,ext) / replace_extension(fname,ext) - returns a version of fname with the file extension changed to ext.
get_mime_for_ext(ext) / get_mime_for_extension(ext) - returns the "standard" MIME type based on the extension found in ext. (ext may be a full filename or just the extension).
get_ext_for_mime(mime) / get_extension_for_mime(mime) - returns the "standard" file extension for the given MIME type.
get_mime_via_magic(file,callback) / get_mime_type_via_magic(file,callback) / get_file_mime_type_via_magic(file,callback) - determines MIME type of file by magic number (ignoring the file extension if any).
get_mime(file,callback) / get_mime_type(file,callback) / get_file_mime_type(file,callback) - determines MIME type of file by magic number or extension. Given a choice, the specific type is selected (e.g., application/json vs. text/plain). The magic-number-based MIME type is used if the two MIME types seem equally specific.
file_is_mime(file,pattern,callback) - calls-back with null, true when the MIME type of file matches pattern.
file_is_pdf(file,callback) - calls-back with null, true when the MIME type of file is application/pdf.
get_mime_to_ext_map() / get_mime_to_extension_map() - returns the MIME-type to "standard" file-extension mapping used for the other methods.
get_ext_to_mime_map() / get_extension_to_mime_map() - returns the file-extension to "standard" MIME-type mapping used for the other methods.
set_mime_to_ext_map(map) / set_mime_to_extension_map(map) - sets the MIME-type to "standard" file-extension mapping used for the other methods. Extensions should NOT contain a leading dot (.). Set to null to restore the default mapping.
set_ext_to_mime_map(map) / set_extension_to_mime_map(map) - sets the file-extension to "standard" MIME-type mapping used for the other methods. Extensions should NOT contain a leading dot (.). Set to null to restore the default mapping.
add_to_mime_to_ext_map(map | mime, ext) / add_to_mime_to_ext_map(map | mime, ext) - adds a collection or a single instance of a mime-to-file-extension mapping to the currently active set.
add_to_ext_to_mime_map(map | ext, mime) / add_to_extension_to_mime_map(map | ext, mime) - adds a collection or a single instance of a file-extension-to-mime-type mapping to the currently active set.

Note that it is not necessarily the case that get_ext_for_mime(get_mime_for_ext( EXT )) == EXT and vice-versa. See the data directory for the default mappings.

Back to Index

IOUtil

pipe_to_file(readable_stream,dest,options,callback) - write a stream to a file. (callback(err,file)).
pipe_to_buffer(readable_stream,callback) - write a stream to a buffer. (callback(err,buffer)).
download_to_file(url,dest,options,callback) - write the contents of a URL to a file. (callback(err,file)).
download_to_buffer(url,callback) - write the contents of a URL to a buffer. (callback(err,buffer)).
download_to_data_uri(url,callback) - convert the contents of a URL to a data-uri. (callback(err,uri)).

Back to Index

L10nUtil

identify_locales(req) - identify a list of [ lang, REGION ] pairs based on the given request object.
expand_locales(locales) - given a list of[ lang, REGION ] pairs, adds [ lang, null ] to this list if otherwise missing.
identify_and_expand_locales(req) - `expand_locales(identify_locales(req))
match_locale(accepted, available, default_value) - given a list of accepted [ lang, REGION ] pairs and a map of lang-region: non-null, returns an acceptable key (or the default_value) when no matching locale is found.
load_l10n_files(dir, options, callback) - loads a set of localization files (JSON + comments) from the specified directory, returning a map of 'name-without-extension': loaded-content. When present options are options passed to FileUtil.ls, notably recurse and pattern (defaulting to false and /^.+.json$/, respectively). Note that a specific lang-REGION's attributes will inherit default values from those in lang (when set), such that if en.json has the key foo but en-US.json does not, the returned en-us object will contain the key foo. Callback signature is (err, map_of_localiation_data)
localize(localization_data, key, args...) - returns the localized value for the sprintf template localization_data[key], passing in the optional args arguments when provided. Returns null if no matching template is found.
- note that a localization entry may be a map from which one specific template will be selected according to the value of the first argument (see unit tests for details).
make_localizer(localization_data) - returns the function of the form fn(key, [args...]) which is equivalent to localize(localization_data, key, [args...]).

LogUtil

Some notes on configuration:
- require('inote-util').LogUtil yields a "singleton" object with a default configuration.
- require('inote-util').LogUtil.init is a function that invokes the LogUtil constructor
- require('inote-util').LogUtil.LogUtil and require('inote-util').LogUtil.constructor provide direct access to the LogUtil constructor.
- Both LogUtil.init and the LogUtil.LogUtil constructor accept a single "configuration" object, a map with the following (optional) attributes:
  - debug - when true LogUtil.[t[p]]debug(...) while write to LogUtil.[t[p]]log(...); otherwise LogUtil.debug produces no output.
  - prefix - when a non-null string is provided it will be prefixed to every log line (after the timestamp and pid identifiers, where applicable).
  - logger - an object that defines the various underlying logging methods; by default this is console. The logger map can be used to override the default behavior by providing any of the following methods:
    - log(...)
    - info(...)
    - warn(...)
    - error(...)
log(...), info(...), warn(...), err(...), error(...) - writes to stdout/stderr or to the equivalent method in the logger object provided to LogUtil.init
tlog(...), tinfo(...), twarn(...), terr(...), terror(...) - prepends a timestamp to each log message (E.g., [2015-05-17T22:03:58.569Z] Hello World!)
tplog(...), tpinfo(...), tpwarn(...), tperr(...), tperror(...) - prepends a timestamp and process ID to each log message (E.g., [2015-05-17T22:03:58.569Z] [p:123] Hello World!)
debug(...), tdebug(...), tpdebug(...) - like log/tlog,tplog but only writes to the log if the debug attribute passed to the constructor is true.

Back to Index

NetUtil

is_port_in_use(port,callback) - attempt to determine whether or not a port is currently in use
get_unused_port(callback) - attempt to obtain an unused port
normalize_url

Back to Index

NumberUtil

round_decimal(value,digits=0) - round a number to the specified number of digits to the right of the decimal point.
is_int(val) - returns true if and only if val is a simple integer (matching /^-?((0)|([1-9][0-9]*))$/).
to_int(val) - returns parseInt(val) when val is a simple integer (matching is_int), null otherwise. (Compare with the default behavior of parseInt, which returns 17 for parseInt("17.93 meters")).
is_float(val) - returns true if and only if val is a simple decimal value (matching /^-?((((0)|([1-9][0-9]*))(\.[0-9]+)?)|(\.[0-9]+))$/).
to_float(val) - returns parseFloat(val) when val is a simple decimal value (matching is_float), null otherwise.

Back to Index

ObjectUtil

get_json_path(json,path...) - fetches that attribute stored at the specified path
get_funky_json_path(json,path...) - fetches that attribute stored at the specified path, handlig the @name and name.$ attributes that some XML-to-JSON parsers produce.
deep_equal(a,b) - performs a deep comparison of the two specified objects; handles arrays, maps, strings, booleans, numbers, null and undefined, results are undefined for other object types; considers null, undefined and missing elements to be equal.
is_true_object(a) - true iff a is a non-null, non-array object for which typeof a == 'object'.
diff_json(a,b) - recursively compares elements of a and b, returning a map describing where the objects differ; handles arrays, maps, strings, booleans, numbers, null and undefined, results are undefined for other object types; considers null, undefined and missing elements to be equal. Some examples:
- {foo:{bar:1}}, {foo:{bar:1}} yields undefined (no difference)
- 3, 3} yields undefined (no difference)
- {foo:{bar:1}}, {foo:{bar:2}} yields {foo:{bar:'c'}} (value of foo.bar has changed.
- {foo:{bar:null}}, {foo:{bar:2}} yields {foo:{bar:'a'}} (value of foo.bar was added.
- {foo:{bar:1}}, {foo:{x:true}} yields {foo:{bar:'d',x:'a'}} (value of foo.bar was deleted and value of foo.x was added).
remove_null(map,recurse=false) - generates a (shallow) clone of the map or array, with null entries removed; when recurse is true the action is repeated for each map or array-value element or attribute
remove_falsey(map,recurse=false) - generates a (shallow) clone of the map, with "falsey" entries removed (see falsey_string); when recurse is true the action is repeated for each map or array-value element or attribute
merge(maps...) - given two or more maps a and b, creates new new map containing the union of elements from each. If a and b share a key, the value in b will overwrite the value in a.
deep_merge(maps...) - just like merge except that sub-maps are recursively merged (e.g., merging {a:{a1:1}} and {a:{a2:2}} yields {a:{a1:1,a2:2}} rather than {a:{a2:2}}).
shallow_clone(obj) - create a shallow clone of the given map or array.
deep_clone(obj) - recursively copy the elements obj into a new map (or array)
object_array_to_map(array,key_field,options={})) - Given a list of objects, creates a map from elt[key_field] to elt for each elt in the array.
- options.transform - an optional function used to transform the value of elt[key_field] before using it as the map key.
- options.duplicates - a string indicating how to handle duplicate keys:
  - "overwrite" - replace the old value with the new value (the default)
  - "stack" - create an array containing both values (in sequence)
  - "merge" - merge the objects using Util.merge(old,new)
  - "skip" - keep the old value and ignore the new one

Back to Index

RandomUtil

random_bytes([count=32,encoding='hex']) - returns count random bytes in the specified encoding.
seed_rng(seed) - returns a new random() function with the specified seed value.
set_rn(rng = Math.random) - sets the random() function used by the RandomUtil methods.
random_hex([count=32,rng]) - returns count random hexadecimal digits ([a-f0-9]) (using the given random number generator if provided).
random_alphanumeric([count=32,rng]) - returns count random digits from the set [a-z0-9] (using the given random number generator if provided).
random_alpha([count=32,rng]) - returns count random digits from the set [a-z] (using the given random number generator if provided).
random_numeric([count=32,rng]) - returns count random digits from the set [0-9] (using the given random number generator if provided).
random_Alphanumeric([count=32,rng]) - returns count random digits from the set [0-9a-zA-Z] (using the given random number generator if provided).
random_Alpha([count=32,rng]) - returns count random digits from the set [a-zA-Z] (using the given random number generator if provided).
random_ALPHA([count=32,rng]) - returns count random digits from the set [A-Z] (using the given random number generator if provided).
random_element(collection,rng) - returns a random element from an array, or [key,value] pair given a map (using the given random number generator if provided).
random_value(min,max,rng) - returns a random float between min and max, inclusive; min defaults to 0, max defaults to 1
randomly_assign(id,categories) - assign the given (non-null) identifier to a randomly selected category such that (A) fgor a given collection of identifiers results will be randomly distributed among the categories and (B) for a given identifier the result will be the same every time the method is called. The categories parameter is optional: when missing, identifiers will be randomly assigned to either true or false; when an value between 0 and 1, identifiers will be assigned to either true or false, with true appearing 100*value % of the time; when an array, identifiers will be assigned to an element of the array with equal probability
shuffle(list) - performs an in-place shuffle of the given list

Back to Index

SimpleCache

new SimpleCache({default_ttl:, default_value:, purge_interval:})
cache.put(key, value, {ttl:})
cache.get(key, {ignore_ttl:, default:}) / cache.set(key, {ignore_ttl:, default:})
cache.get_put(key, value, {ttl:, ignore_ttl:, default:}) / cache.get_set(key, value, {ttl:, ignore_ttl:, default:})
cache.clear(key)
cache.get_clear(key, {ignore_ttl:, default:})
cache.clear_all()
cache.clear_matching(/regexp/)
cache.keys({ignore_ttl:})
cache.values({ignore_ttl:})
cache.pairs({ignore_ttl:})

Back to Index

Stopwatch

A simple utility that can be used to track and report the time it takes to do some thing in your JavaScript code.

Basic Use

var SW = require('inote-util').Stopwatch;
var timer = SW.start();
// ...do something...
timer.stop();
console.log("Start Time:  ",timer.start_time);
console.log("Finish Time: ",timer.finish_time);
console.log("Elapsed Time:",timer.elapsed_time);

Wrapped (Synchronous)

timer = SW.time( some_method );
console.log("some_method took",timer.elapsed_time,"millis to complete.");

"Cookies"

The start and time methods accept an optional map of attributes that will be bundled with the returned timer. For example:

timer = SW.start({label:"foo"});
// ...do something...
timer.stop();
console.log(timer.label," Start Time:  ",timer.start_time);
console.log(timer.label,"Finish Time: ",timer.finish_time);
console.log(timer.label,"Elapsed Time:",timer.elapsed_time);

Back to Index

StringUtil

trim(str) - equivalent to String.trim() save that str can be null.
is_blank(str) - true whenever str is empty, composed entirely of whitespace, null or undefined.
isnt_blank(str) - opposite of is_blank()
blank_to_null(str) - given a "blank" string, returns null. Given an object (map), removes any top-level "blank" attributes.
truncate(str,width,marker='…') - a minimally "smart" truncation that attempts to truncate a string at a word boundaries. The specified marker will be added if and only if the string was actually truncated.
escape_for_json(str) - escapes a (possibly null) string for use as literal characters in a JSON string.
escape_for_regexp(str) - escapes a (possibly null) string for use as literal characters in a regular expression.
escape_for_bash(param,esc_specials=false) - escapes a (possibly null) string for use as command-line parameter. When esc_specials is false (the default) parameters such as <, >, &&, etc. are not escaped.
sanitize_for_html
json_or_null
StringUtil.sanitize_for_sql_like
truthy_string(str) - true if the given string is t, true, y, yes, on, 1, etc.
falsey_string(str) - true if the given string is f, false, no, off, 0, etc.
lpad(value,width,pad) - adds pad characters to the beginning of value until value is width characters long. (Also accepts arrays, see ArrayUtil.lpad, which is an identical method.)
rpad(value,width,pad) - adds pad characters to the end of value until value is width characters long. (Also accepts arrays, see ArrayUtil.rpad, which is an identical method.)
shuffle(list) - performs an in-place shuffle of the given list

Back to Index

Util

get_funky_json(json,keys...) -t he call get_funky_json(json, "foo", "bar") will return Xyzzy for { foo: { bar: "Xyzzy" } } and { foo: { bar: {$:"Xyzzy" } } } and { foo: { @bar: "Xyzzy" } }, etc. This is used to mask the differences between certain XML-to-JSON translators. This method is also aliased as gfj.
version_satisfies([version_string,]range_string) - returns true if the given version_string satisfies the semver criteria specified in range_string. When version_string is omitted, the Node version (the value of process.version) is used. E.g., if(!Util.version_satisifes(">=0.12.3")) { console.error("Expected version 0.12.3 or later."); }.
slow_equals(a,b) - constant-time comparison of two buffers for equality.
compare(a,b) - a minimally-smart comparision function (allows null, uses localeCompare when available, folds case so that both A and a appear before B, etc.).
field_comparator(field,use_locale_compare=false) - returns a comparator (function(a,b)) that compares two maps on the field attribute.
path_comparator(path,use_locale_compare=false) - like field_comparator, but path may be an array of fields that will be interpreted as nested-attributes. (E.g., ["foo","bar"] compares a.foo.bar with b.foo.bar.)
descending_comparator(comparator) / desc_comparator(comparator) - reverses the order of the given comparator.
composite_comparator(list) - generates a comparator that first compares elements by list[0] then (if equal) list[1] and so on, until a non-equal comparison is found or we run out of comparators.
handle_error(err[,callback,throw_when_no_callback=true]) - if err is not null, invokes callback(err) or throw err as appropriate. Returns true if an error was encountered, false otherwise. (function my_callback(err,other,stuff) { if(!handle_error(err,callback)) { /* keep going */ } })
uuid(val,generate=false) - normalize val to all a lower-case, no-dash version of a UUID. If generate is true, generate an new UUID when given a null val, otherwise returns null in that scenario. (DEPRECATED)
- As of v1.8.3, when called with no arguments uuid() generates a UUID value, while uuid(null) retains the original behavior.
normalize_uuid(val,generate=false) - non-deprecated replacement for uuid(val,generate).
make_uuid(val), make_uuid_v1(val), make_uuid_v4(val) - return a normalized version of the specified UUID, creating a new one if none is proivded.
pad_uuid(val,generate=false) - normalize val to all a lower-case, with-dashes version of a UUID. If generate is true, generate an new UUID when given a null val, otherwise returns null in that scenario.
b64e(buf,encoding='utf8') / Base64.encode(buf,encoding='utf8') - Base64 encode the given buffer.
b64d(buf,encoding='utf8') / Base64.decode(buf,encoding='utf8') - Base64 decode the given buffer.

Back to Index

WebUtil

remote_ip(req,name,default_value) - attempts to discover the proper "client IP" for the given request using various approaches.
param(req, name, default_value) - replaces the now deprecated req.param(name,default_value) found in Express.js. name can also be an array of names to check.
map_to_qs(map) (also map_to_query_string(map)) - convert the given map to a string of the form name1=value1&name2=value2&etc. for which names and values are properly URL-encoded. When a key maps to an array the key is appended multiple times (key=value1&key=value2&etc.).
append_qs(url,map) / append_qs(url,qs_fragment) / append_qs(url,name,value) (also append_query_string())

Back to Index

WorkQueue

WorkQueue implements a basic priority queue for asynchronous tasks.

Users may add "tasks" to the work queue, which will be asynchronously executed according to priority order.

Each task consists of:

a function
optionally, an array of arguments to the function
optionally, a priority for the task
optionally, a callback function to invoke when the task is complete

The final parameter to the task-function will always be the WorkQueue's done function. Users must call this function to indicate that the task is complete (or, optionally throw an exception).

Any arguments passed to the done function will in turn be passed to the callback function registered with the task (if any).

Example

var WorkQueue = require("inote-util").WorkQueue;
var wq = new WorkQueue();

// A task
function taskOne(a,b,cb) {
  console.log("taskOne is executing. a=",a,"b=",b);
  cb(1,false,a);
}
var argsOne = [ "A", "B" ];

// Another task
function taskTwo(cb) {
  console.log("taskTwo is executing.");
  cb(2,true,null);
}

// A callback that will be invoked after each task
function afterTask(num, exit, a) {
  console.log("Task number",num,"is completed. a=",a);
  if (exit) {
    wq.stop_working();
  }
}

wq.enqueue_task(taskOne, argsOne, afterTask);
wq.enqueue_task(taskTwo, afterTask);

wq.start_working();

When run, that script generates the following output:

taskOne is executing. a= A b= B
Task number 1 is completed. a= A
taskTwo is executing.
Task number 2 is completed. a= null

Events

The WorkQueue is also an EventEmitter, with the following events:

work-beginning - called when the queue starts processing. Arguments: the WorkQueue instance.
task-enqueued - called when a task is added to the queue. Arguments: the WorkQueue instance and the task ({priority,method,args,callback}).
task-dequeued - called when a task is selected for processing. Arguments: the WorkQueue instance and the task ({priority,method,args,callback}).
task-completed - called when a task is done (and didn't throw an uncaught exception.) Arguments: the WorkQueue instance, the task and an array of arguments passed to the done function..
error - called when an uncaught exception is encountered while processing the task. Arguments: the WorkQueue instance, the task and the error.
busy - called when a task is deferred because the maximum number of tasks are currently active. Arguments: the WorkQueue instance.
work-ending - called when the queue stops processing. Arguments: the WorkQueue instance.

Methods

new WorkQueue([options]) - create a new WorkQueue instance. Options:
- priority - the default priority for tasks in the queue (when none is specified when the task is added). Defaults to 5. Larger numbers are executed before smaller numbers.
- interval - the time (in milliseconds) between "polling" the queue for new tasks to execute. Defaults to 200.
- fuzz - a floating point value that will be used to "fuzz" the interval between polling runs. When fuzz is a non-zero float, a random value between -1 × fuzz × interval and fuzz × interval will be added to the interval. This is used to avoid synchronization when several WorkQueues are launched with the same configuration. (E.g., when a node app that uses the WorkQueue is launched in a cluster.) Defaults to 0.1.
- workers - the maximum number of tasks that can be active at one time. Defaults to 1.
pending_task_count() - returns the number of tasks in the queue waiting to be executed.
active_task_count() - returns the number of tasks from the queue currently being executed.
enqueue_task:(method[,args][,priority][,callback]) - add a task the the queue.
start_working([options]) - start queue processing. The options parameter may contain interval or fuzz attributes, which will override those provided in the construction for the duration of this processing.
stop_working() - stop queue processing. Note that if a queue is started but not stopped, a function will be called roughly every interval milliseconds via JavaScript's setInterval method.

Back to Index

ZipUtil

zip(wd,zipfile,inputs,callback) - creates (or appends to) zipfile, adding the file or array of files in inputs. When wd is specified the action will take place relative to that directory.
unzip(wd,zipfile,dest,callback) - unzips zipfile into the specified dest.
contents(zipfile,callback) - obtains a list of files within zipfile.

Back to Index

Errors

TimeoutError
ExceptionThrownError

Back to Index

Up to ToC

Installing

From Source

The source code and documentation for inote-util is available on GitHub at intellinote/inote-util. You can clone the repository via:

git clone git@github.com:intellinote/inote-util

Assuming you have node installed, you can then install the remaining dependencies by running npm install from the root repository directory.

A Makefile is provided. Run make install to install. Run make test to run the unit test suite. Run make help for a list of other useful Make targets.

From npm

inote-util is deployed as an npm module under the name inote-util. Hence you can install a pre-packaged version with the command:

npm install inote-util

and you can add it to your project as a dependency by adding a line like:

"inote-util": "latest"

to the dependencies or devDependencies part of your package.json file.

Up to ToC

Licensing

The inote-util library and related documentation are made available under an MIT License. For details, please see the file LICENSE.txt in the root directory of the repository.

Up to ToC

How to contribute

Your contributions, bug reports and pull-requests are greatly appreciated.

We're happy to accept any help you can offer, but the following guidelines can help streamline the process for everyone.

You can report any bugs at github.com/intellinote/inote-util/issues.
- We'll be able to address the issue more easily if you can provide an demonstration of the problem you are encountering. The best format for this demonstration is a failing unit test (like those found in ./test/), but your report is welcome with or without that.
Our preferred channel for contributions or changes to the source code and documentation is as a Git "patch" or "pull-request".
- If you've never submitted a pull-request, here's one way to go about it:
  1. Fork or clone the repository.
  2. Create a local branch to contain your changes (git checkout -b my-new-branch).
  3. Make your changes and commit them to your local repository.
  4. Create a pull request as described here.
- If you'd rather use a private (or just non-GitHub) repository, you might find these generic instructions on creating a "patch" with Git helpful.
If you are making changes to the code please ensure that the unit test suite still passes.
If you are making changes to the code to address a bug or introduce new features, we'd greatly appreciate it if you can provide one or more unit tests that demonstrate the bug or exercise the new feature.
This repository follows the "git flow" branching convention. The master branch contains snapshots of each stable release. New development occurs within the develop branch. We'd prefer most pull-requests to be based off the develop branch.

Please Note: We'd rather have a contribution that doesn't follow these guidelines than no contribution at all. If you are confused or put-off by any of the above, your contribution is still welcome. Feel free to contribute or comment in whatever channel works for you.

Up to ToC