Taglex NPM | npm.io

TagLex

TagLex is library containing streaming Lexers and Parsers for processing custom mark-up languages. It makes writing a Markdown-like language parser very easy. It also facilitates writing parsers for strict data formats.

Installation

npm install taglex

Usage

Check out the Examples for an example full Markdown-esque language, or see below for mini-language examples.

Simple example

var taglex = require('taglex');
var sys = require('sys');

var ruleset = new taglex.TagRuleSet({ ignore_case: true });
ruleset.add_tag({
    name: 'italic',
    open: '*',
    close: '*',
    parents: ['root'],
    aliases: [['_', '_'], ['i:', ':i']],
    payload: {start: '<i>', finish: '</i>'}
});

var parser = ruleset.new_parser();
parser.on('tag_open', function (payload, token) {
    sys.print(payload.start);
});

parser.on('text_node', function (text) {
    sys.print(text.replace(/</g, "&lt;")); // escape
});

parser.on('tag_close', function (payload, token) {
    sys.print(payload.finish);
});

parser.write("This is an *example* of a small I:regular");
parser.write(" language:I");

// Would output:
// This is an <i>example</i> of a small <i>regular language</i>

Tag hierarchy example

TagLex is also capable of parsing context-free grammars:

var ruleset = new taglex.TagRuleSet();
ruleset.add_tag({
    name: 'table',
    open: '{{{', close: '}}}',
    ignore_text: true,
    parents: ['root'],
    payload: {start: '<table>', finish: '</table>'}
});

ruleset.add_tag({
    name: 'row',
    ignore_text: true,
    open: '[', close: ']',
    parents: ['table'],
    payload: {start: '<tr>', finish: '</tr>'}
});

ruleset.add_tag({
    name: 'cell',
    open: '[', close: ']',
    parents: ['row'],
    payload: {start: '<td>', finish: '</td>'}
});

// to make it context-free, a tag that can contain itself:
ruleset.add_tag({
    name: 'i',
    open: '[', close: ']',
    parents: ['i', 'cell'],
    payload: {start: '<i>', finish: '</i>'}
});

/* [... parser set up as before ...] */

parser.write("Outside the table I can freely use [] characters.\n");
parser.write("Here is a table example:\n{{{ (ignored text)");
parser.write("[ [ cell 1 ] [[[ cell 2 ]]] [ 3 ] ]\n");
parser.write("[ [ cell 4 ] [ cell 5 ] [ 6 ] ]");
parser.write("}}}");

// Would output (wrapped):
// Outside the table I can freely use [] characters.
// Here is a table example:
// <table><tr><td> cell 1 </td><td><i><i> cell 2 </i></i></td><td> 3 </td></tr>
// <tr><td> cell 4 </td><td> cell 5 </td><td> 6 </td></tr></table>

Speed

I haven't benchmarked it, or carefully looked at complexity, but to give you a broad idea of what to expect:

Compile step is at least O(n^2) both for memory and CPU, with n = number of tags.
Render step should be very fast as it relies on searching the input string by a single regular expression (per context). The slowest feature is the "stack collapse" feature.

Anti-features

TagRuleset aliases are counter-intuitive. Presently, they can be mixed and matched. Assume that in the future this will change, that opening with one alias can only close with that alias.
The "stack collapse" feature (enabled with the option to add_tag "force_close") sometimes splits TEXT_NODE emissions, typically this is a harmless bug. This feature in general is needlessly complex and could use a re-write.
Poor documentation: TagLex documentation could use a lot of work. In the mean time, check out examples and tests.js to see many more examples of what you can do.
Very large number of heavily interacting tags (e.g. where tag nesting is a complete graph, and sloppy tag closes apply everywhere, such as a fault-tolerant HTML parser) might mean a slow compile step and unnecessarily larger memory footprint (lots of n^2 operations)