libharmonia v0.0.3
libharmonia
An opinionated microservices framework for node.js and docker
Part of harmonia.systems, libharmonia
is the foundation of this opinionated system. It's role is
to provide a common inter-service messaging platform, deeply integrated with logging and tracing.
The harmonia.systems framework is one that explicity relys on pattern matching messages sent between services over a message queue.
Usage
libharmonia
is written in Typescript, and can also be used with ES6/ES7 code.
You'll have the most fun with async/await
, with either Typescript or Babel. Keep an eye
on the NodeJS production releases for when a stable async/await
implementation hits NodeJS core.
In harmonia there are two roles that a system can perform. Sender
and Listener
. By default, all
services can send out messsages as a Sender
, and with extra configuration they can also
listen for messages sent to them as a Listener
.
Let's create both in both Typescript and ES7 code.
Sender
Our sender is going to send a math problem to a service called math
that can return the
solution.
import {
Harmonia,
MessageResponse,
} from 'libharmonia';
// Initialise Harmonia
const harmonia: Harmonia = new Harmonia({
amqp: {
host: 'mq',
port: 5672,
username: 'admin',
password: 'secret',
virtualHost: 'harmonia',
},
role: 'math-tester',
});
// Lets create some numbers for this problem
const x: number = 12;
const y: number = 83;
const z: number = 32;
// We wrap our harmonia.message calls in harmonia.request
// This is essential to take advantage of the built in request tracing
// harmonia.request is an async function that takes in a callback,
// and passes a fresh requestId as the only parameter.
// Then, all requests made inside our callback function, will
// be linked together in our logging, allowing you to visualise
// the flow of requests made throughout the system.
const mathSolution: number = await harmonia.request(
// harmonia.request takes a callback function, which passes a fresh requestId
// as it's only parameter
async (requestId: string) => {
// harmonia.message is another async function that takes our requestId as
// it's first parameter and then an object that comprises the message as
// it's second parameter.
// Our message body needs to have a `service` (where the message is going),
// an `action` (what action/method to perform on our payload)
// and `payload` (an object).
const sumResult: MessageResponse = await harmonia.message(
requestId,
{
service: 'math',
action: 'sum',
payload: {
x,
y,
},
});
// Here we're making another request.
const productResult: MessageResponse = await harmonia.message(
requestId,
{
service: 'math',
action: 'product',
payload: {
x: sumResult.payload,
y: z,
},
});
// Finally we're returning the payload of productResult
// this will become the value of mathSolution
return productResult.payload;
}
);
// Log our final result
console.log(`(${x} + ${y}) * ${z} = ${mathSolution}`);
// This should log:
// (12 + 83) * 32 = 3,040
API
Error handling
All of the public functions with harmonia
except the constructor new Harmonia()
are async functions (Promises).
All the public functions will throw an error if something went wrong, thus they must all be wrapped in
a try ... catch
statement, otherwise your errors may be swallowed.
new Harmonia(options)
The harmonia constructor lets you configure how this instance of harmonia is going to connect to your infrastructure.
Harmonia can connect to either an AMQP message queue like RabbitMQ or a lightweight high performance messaging bus like NATS.
It's also possible to supply your own or (third-party) sender and listener implementations, to connect
with a different messaging system. The sender and listener will both receive the same messengerConfig
,
and use the same messaging system. You can't send messages with AMQP, and listen for them with NATS.
At least, not within the same instance, you could setup two harmonia instances with different messaging
systems if you needed a bridge.
Harmonia by default will log to Graylog2 with GELF
, but again, you can pass through your own logging
implementation if you need to use something else. Graylog2 was chosen as it's a great open source
logging project, that works out of the both for debugging messages, but can also forward messages to
other systems.
Usage
import {
Harmonia,
} from 'libharmonia';
// Always try..catch as errors are thrown not returned
try {
// Initialise Harmonia
const harmonia: Harmonia = new Harmonia({
// Define your services name
// This is a global namespace, only one repository should use it.
// It's expected that you will run multiple containers with this repo,
// but that repos will all have a unique service name.
// Ulitmately this is how meesages are routed.
service: 'math',
// Provide connection details to the messenger
messengerConfig: {
type: 'ampq', // amqp or nats
host: 'mq',
port: 5672,
username: 'admin',
password: 'secret',
virtualHost: 'harmonia',
},
// (optional) you can provide a custom or third party sender and listener if
// you need to use something besides amqp or nats
// Look at src/modules/messenger/sender.ts and src/modules/messenger/listener.ts for
// implementation details
sender: () => {},
listener: () => {},
// Provide the connection information for the logging service
loggerConfig: {
type: 'graylog2', // default implementation is graylog2
servers: [
{
host: 'harmonia-graylog',
port: 1984
},
],
hostname: 'dev', // the name of this host
// (optional, default: os.hostname())
facility: 'Node.js', // the facility for these log messages
// (optional, default: "Node.js")
bufferSize: 1350, // max UDP packet size, should never exceed the
// MTU of your system (optional, default: 1400)
}
// (optional) you can pass a custom logger which will be passed loggerConfig
// Look at src/modules/logger.ts for implementation details
logger: () => {}
// (optional) Provide the connection details for the database connection
databaseConfig: {
type: 'rethinkdb',
}
// (optional) provide your own or a third party database implementation
database: () => {}
});
} catch (harmoniaError){
// If we could not start for any reason we should crash.
console.log('Harmonia failed to start', harmoniaError)
process.exit(1);
}
harmonia.request(callback)
Used to wrap harmonia.message
calls. This function
ensures a fresh requestId can be passed to harmonia.message
, it also clears out the internal state
that tracks the previous message -- allowing for detailed tracing of requests and messages.
Usage
// Always try..catch as errors are thrown not returned
try {
const request: MessageResponse = await harmonia.request(
// harmonia.request takes a callback function,
// which passes a fresh requestId as it's only parameter
async (requestId: string) => { // note it's an async function
// call harmonia.message Here
}
);
} catch(requestError){
// handle the requestError
}
harmonia.message(requestId, message)
This is an async
function (a Promise) that will return the response from the remote service.
Usage
// Always try..catch as errors are thrown not returned
try {
const messageResult: MessageResponse = await harmonia.message(
requestId,
{
// The service you are sending the message to
service: 'math',
// The action you need the service to perform on your payload
action: 'sum',
// You payload, in the format required by the action on the chosen service
payload: {
x: 12,
y: 32,
},
});
// Use the result
console.log(messageResult.payload);
} catch(messageError){
// handle messageError
}
The response from harmonia.message
will always match MessageResponse
which you can see below.
MessageResponse
interface MessageResponse {
// The response
payload: any;
// Mainly internal metadata, but you may want some of it
metadata: MessageMetadata;
// if the data payload contains sensitive information that should not be logged
// we indicate it here, and it should never be stored, it MUST be zeroed before logging
sensitiveKeys?: string[];
}
harmonia.addListener(pattern, callback(message, error))
To register a listener you add the action and callback function with a call to harmonia.addListener
.
The service is set in the constructor of the harmonia
instance, and can't be changed here.
First create an async
function which takes the message
and error
as arguments.
Then call harmonia.addListener
, specify the pattern to match on, and then the callback function.
Usage
async function sum(message: number, error: any) {
return message.payload.x + message.payload.y;
}
harmonia.addListener({ action: 'sum' }, sum);
You can also pass a fat-arrow function callbacks as well. Though these will be much harder to unit test, and therfore are not reccomended.
harmonia.addListener(
{ action: 'sum' },
async (message: number, error: any) => {
return message.payload.x + message.payload.y;
}
);
We use patrun
for pattern matching. Generally speaking the most specific pattern will always win.
Error handling in listener callbacks
The function that calls your callback will try ... catch
your function for any exceptions and pass
them back as an error. You don't explicity need to catch errors inside your callback, if you don't need
to handle it.
Install
Generally you'll find it easier to start from one of our starterkits, as Harmonia is designed to be a framework to build upon.
TODO build and add a starter kit.
Standalone
With npm installed, run
$ npm install libharmonia
Or with yarn
$ yarn add libharmonia
Acknowledgments
seneca.js
is a similar an far less opinionated module, that was the primary inspiration
for libharmonia
patrun
for an excelent pattern matching library.
See Also
License
ISC License (ISC)
Copyright (c) 2017, Owen Kelly owen@owenkelly.com.au
Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.