1.11.0 • Published 1 week agoCLI

ldn-inbox-server

Licence

MIT

Version

1.11.0

Deps

Size

89 kB

Vulns

Weekly

Summary Dependency Versions

ldn-inbox-server

An experimental LDN inbox server for Event Notification messages.

Install

yarn add ldn-inbox-server

Example

Create required directories

mkdir config inbox public

Start the server:

npx ldn-inbox-server start-server --port 8000

Send a demonstration Event Notifications message:

curl -X POST -H 'Content-Type: application/ld+json' --data-binary '@examples/offer.jsonld' http://localhost:8000/inbox/

Start an inbox handler with a demo handler (that creates an Accept message in the ./outbox).

npx ldn-inbox-server handler @inbox -hn ./handler/accept_notification_handler.js

Start an outbox handler that send the notifications that are available outbox:

npx ldn-inbox-server handler @outbox -hn ./handler/send_notification_handler.js

Environment

LOG4JS : log4js logging level
LDN_SERVER_HOST : LDN inbox host
LDN_SERVER_PORT : LDN inbox port
LDN_SERVER_INBOX_URL : LDN inbox url (path)
LDN_SERVER_INBOX_PATH : LDN inbox path
LDN_SERVER_ERROR_PATH : LDN error path
LDN_SERVER_OUTBOX_PATH : LDN outbox path
LDN_SERVER_PUBLIC_PATH : public (HTML) path
LDN_SERVER_JSON_SCHEMA : notification JSON validation schema
LDN_SERVER_BASEURL : baseurl of the LDN inbox server
LDN_SERVER_INBOX_GLOB : glob of files to process in inbox directory
LDN_SERVER_HAS_PUBLIC_INBOX : if true, then public read access is allowed on the inbox
LDN_SERVER_HAS_WRITABLE_INBOX : if true, then public write access is allowed on the inbox
LDN_SERVER_MAX_UPLOAD_SIZE : max POST body size in bytes (0 or unset = unlimited); fallback when no reverse proxy enforces a limit
LDN_SERVER_RATE_LIMIT : max POST requests per client IP per window (0 or unset = unlimited); fallback when no reverse proxy enforces a limit
LDN_SERVER_RATE_WINDOW : rate limit window in seconds (default 60)
LDN_SERVER_LOCKDIR : directory to store optional lock files for handler

Config files (*.json, *.json5, *.yaml) are interpolated with environment variables when loaded: ${VAR} is replaced with process.env.VAR, and ${VAR:-default} falls back to default when VAR is unset or empty. An unset variable with no default is left as the literal ${VAR}. This lets one set of config files serve multiple deployments — e.g. an actor id "${WEBID_BOT}" or a filter value "${WEBID_BOT:-http://localhost:3002/profile/card#me}". JSONPath expressions like $.origin.id are untouched (only ${...} is substituted).

Multiple inboxes

Instead of a single inbox, multiple inboxes can be configured by adding a JSON configuration file to the installation. The JSON file should contain a registry entry with contains an array of inbox configuration. An example:

{
  "registry": [
        { "path": "inbox/.*" , 
          "with": {
            "url": "inbox/",
            "inbox": "./inbox",
            "inboxPublic": 1,
            "inboxWritable": 1,
            "schema": "./config/schema1.json"
        }},
        { "path": "inbox2/.*" , 
          "with": {
            "url": "inbox2/",
            "inbox": "./inbox2",
            "inboxPublic": 1,
            "inboxWritable": 0,
            "schema": "./config/schema2.json"
        }}
    ]
}

Extend

Server extensions are possible by providing custom inbox and notification handlers. E.g.

npx ldn-inbox-server handler @inbox -hn handler/my_handler.js

Or, in JavaScript:

const { handle_inbox } = require('ldn-inbox-handler');

main();

async function main() {
    await handle_inbox('./inbox', {
        'inbox': './inbox',
        'outbox': './outbox',
        'public': './public',
        'error': './error',
        'batch_size': 5,
        'glob': '^.*\\.jsonld
with my_handler.js :
async function handle({path,options}) {
    //...

    return { path, options, success: true };
}

module.exports = { handle };
Hints
A handler can be started on any directory. E.g. a workflow might be:

have an "inbox" handler to validate incoming LDN messages
valid LDN messages will be saved into the "accepted" box
invalid LDN messages will be saved into the "error" box
have an "accepted" handler to process valid LDN messages
processed LDN messages will end up in the "outbox" box
invalid processing will be saved into the "error" box

Handlers
Accept handler
A handler that creates for an incoming notification an Accept notification in the @outbox.
Eventlog handler
A handler that updates an event log with the incoming notification. 
Requires configuration properties:

log: the path to the event log (starting from the public directory)
dir: the path to a container to store the events (starting from the public directory)

The log and dir path may contain a @artifact(:strip)?@ directive to fill in the
path of the current artifact. The :strip filter is used to strip an artifact path of a file extension. E.g. path/artifact.html becomes path/artifact when using a :strip.
Json Path handler
A handler that accepts a notifiction when it matches one or more JSON paths
Requires configuration properties:

anyOf : an array of json path matchers, the combination should be interpreted as a logical OR.
every json path matcher is an array of single matchers, the combination should be interpered as a logical AND.



A single matcher needs two properties:

path : a json path
value : a value

The json path matches when one of:

On the json path an array is found and the value is included in the array
On the json path a string or number is found and the value is equal to this string or number

Multi handler
A handler/multi_notification_handler.js is available to start multiple handler for each notification messages. The handlers to start are specified in a configuration file that can be passed via the config parameter of an handle_inbox. For the commmand line tool bin/ldn-inbox-server the default location of such config file is config/inbox_config.json when processing an @inbox, and config/outbox_config.json when processing an @outbox.
The multi handler requires a configuration file with properties notification_handler.multi.handlers which is an array of array. Each outer array defines a workflow: a list of handlers that should be executed sequentially on a notification until one handler returns a success=false response or the last handler returns a success=true response, or a handler returns break=true (which will skip all other steps in a workflow). The multi handler is successful when at least one workflow could be completed with success.
Each handler is defined by a hash containing as id property the path to the handler and optionally other property keys that will be passed to the handlers in the config section.
Optionally when a fallback handler is defined as option it will be attempted when a handler in a sequence returns a false response. See On Error handler.
Optionally, when a handler configuration includes a $lock property set to true, the handler will generate a lock file to ensure that only a single instance of the handler runs within an LDN inbox processing system.
On Error handler
A handler that sets the fallback for a workflow sequence.
Requires configuration properties:

handler : handler configuration to start when a workflow results in an error

Offer memento handler
A handler to Offer an event log to a memento server.
Requires configuration properties:

actor: the LDN+AS2 actor to use in a notification
target: the LDN+AS2 target to use in a notification

Send notification handler
A handler to send notification that live in the @outbox via the LDN protocol to the LDN target.
If the environoment DEMO_MODE=NO_NOTIFICATIONS is set, no real notifications will be sent.
Type handler
A handler that accepts any notification with a type that matches one of the anyOf types.
Valid artifact handler
A handler that validates the incoming notification and checks if the object or context contains an artifact that is part of the public resources. See 'Artifact support' below.
Generates the following options keys:

artifact.id : the URL of the artifact
artifact.path : the local path to the artifact

Valid event log handler
A hander that validates the incoming notification and checks if the object or context contains an event log that is part of the public resources.
Generates the following options keys:

eventlog.id : the URL of the event log
eventlog.path : the local path of the event log

Artifact support
This code base contains Event Notifications support for Data Node artifacts. See the examples
in public/artifacts-example. Move this directory tp public/artifacts to get a running example.

Each artifact requires at least a .meta file with the X-Artifact header set to true to be recognized by the software as an artifact
Each artifact should update the Link-Template header in the .meta file
The config files in config/inbox_config.json and config/outbox_config.json define the location of the possible event logs for the artifact

API
getLogger()
Returns a LOG4JS logger instance.
fetchOriginal(url)
Resolve the url and return the textual body.
backOff_fetch(url,options)
Return the result of fetch(url,options) (tries many times untill the server responds).
sendNotification(url,payload,options)
Send to url the payload uptionally a fetch can be provided in the options.
moveTo(oldPath, newPath)
Move a file from an oldPath to a newPath .
parseAsJSON(path)
Parse the path into a JSON document (or return null when failed).
generateId()
Generate a uuid URN.
generatePublished()
Generate a ISO8601 date time string.
parseConfig(path)
Parse a path containing .json | .jsonld | .json5 | .yaml | .yml into a
JavaScript object.
Docker
Pull 
docker pull hochstenbach/ldn-inbox-server:v0.0.1
docker run -p 8000:8000 hochstenbach/ldn-inbox-server:v0.0.1
Post a notification:
curl -X POST -H 'Content-Type: application/ld+json' --data-binary '@examples/offer.jsonld' http://localhost:8000/inbox/
Check the inbox: http://localhost:8000/inbox/
See also

mellon-server

,
        'config': './config/inbox_config.json',
        'notification_handler': 'handler/my_handler.js' 
    });
}

with __INLINE_CODE_33__ :

__CODE_BLOCK_9__

Hints

A handler can be started on any directory. E.g. a workflow might be:

have an "inbox" handler to validate incoming LDN messages
valid LDN messages will be saved into the "accepted" box
invalid LDN messages will be saved into the "error" box
have an "accepted" handler to process valid LDN messages
processed LDN messages will end up in the "outbox" box
invalid processing will be saved into the "error" box

Handlers

Accept handler

A handler that creates for an incoming notification an __INLINE_CODE_34__ notification in the __INLINE_CODE_35__.

Eventlog handler

A handler that updates an event log with the incoming notification.

Requires configuration properties:

__INLINE_CODE_36__: the path to the event log (starting from the __INLINE_CODE_37__ directory)
__INLINE_CODE_38__: the path to a container to store the events (starting from the __INLINE_CODE_39__ directory)

The __INLINE_CODE_40__ and __INLINE_CODE_41__ path may contain a __INLINE_CODE_42__ directive to fill in the path of the current artifact. The __INLINE_CODE_43__ filter is used to strip an artifact path of a file extension. E.g. __INLINE_CODE_44__ becomes __INLINE_CODE_45__ when using a __INLINE_CODE_46__.

Json Path handler

A handler that accepts a notifiction when it matches one or more JSON paths

Requires configuration properties:

anyOf : an array of json path matchers, the combination should be interpreted as a logical __INLINE_CODE_47__.
- every json path matcher is an array of single matchers, the combination should be interpered as a logical __INLINE_CODE_48__.

A single matcher needs two properties:

path : a json path
value : a value

The json path matches when one of:

On the json path an array is found and the value is included in the array
On the json path a string or number is found and the value is equal to this string or number

Multi handler

A __INLINE_CODE_49__ is available to start multiple handler for each notification messages. The handlers to start are specified in a configuration file that can be passed via the __INLINE_CODE_50__ parameter of an __INLINE_CODE_51__. For the commmand line tool __INLINE_CODE_52__ the default location of such config file is __INLINE_CODE_53__ when processing an __INLINE_CODE_54__, and __INLINE_CODE_55__ when processing an __INLINE_CODE_56__.

The multi handler requires a configuration file with properties __INLINE_CODE_57__ which is an array of array. Each outer array defines a workflow: a list of handlers that should be executed sequentially on a notification until one handler returns a __INLINE_CODE_58__ response or the last handler returns a __INLINE_CODE_59__ response, or a handler returns __INLINE_CODE_60__ (which will skip all other steps in a workflow). The multi handler is successful when at least one workflow could be completed with success.

Each handler is defined by a hash containing as __INLINE_CODE_61__ property the path to the handler and optionally other property keys that will be passed to the handlers in the __INLINE_CODE_62__ section.

Optionally when a __INLINE_CODE_63__ handler is defined as option it will be attempted when a handler in a sequence returns a __INLINE_CODE_64__ response. See On Error handler.

Optionally, when a handler configuration includes a __INLINE_CODE_65__ property set to __INLINE_CODE_66__, the handler will generate a lock file to ensure that only a single instance of the handler runs within an LDN inbox processing system.

On Error handler

A handler that sets the __INLINE_CODE_67__ for a workflow sequence.

Requires configuration properties:

__INLINE_CODE_68__ : handler configuration to start when a workflow results in an error

Offer memento handler

A handler to __INLINE_CODE_69__ an event log to a memento server.

Requires configuration properties:

__INLINE_CODE_70__: the LDN+AS2 __INLINE_CODE_71__ to use in a notification
__INLINE_CODE_72__: the LDN+AS2 __INLINE_CODE_73__ to use in a notification

Send notification handler

A handler to send notification that live in the __INLINE_CODE_74__ via the LDN protocol to the LDN __INLINE_CODE_75__.

If the environoment __INLINE_CODE_76__ is set, no real notifications will be sent.

Type handler

A handler that accepts any notification with a type that matches one of the __INLINE_CODE_77__ types.

Valid artifact handler

A handler that validates the incoming notification and checks if the __INLINE_CODE_78__ or __INLINE_CODE_79__ contains an artifact that is part of the __INLINE_CODE_80__ resources. See 'Artifact support' below.

Generates the following options keys:

artifact.id : the URL of the artifact
artifact.path : the local path to the artifact

Valid event log handler

A hander that validates the incoming notification and checks if the __INLINE_CODE_81__ or __INLINE_CODE_82__ contains an event log that is part of the __INLINE_CODE_83__ resources.

Generates the following options keys:

eventlog.id : the URL of the event log
eventlog.path : the local path of the event log

Artifact support

This code base contains Event Notifications support for Data Node artifacts. See the examples in __INLINE_CODE_84__. Move this directory tp __INLINE_CODE_85__ to get a running example.

Each artifact requires at least a __INLINE_CODE_86__ file with the __INLINE_CODE_87__ header set to __INLINE_CODE_88__ to be recognized by the software as an artifact
Each artifact should update the __INLINE_CODE_89__ header in the __INLINE_CODE_90__ file
The config files in __INLINE_CODE_91__ and __INLINE_CODE_92__ define the location of the possible event logs for the artifact

Post a notification:

__CODE_BLOCK_12__

Check the inbox: http://localhost:8000/inbox/

Keywords

ldn event notifications