jce-lexer v0.1.0

node-jce-lexer

Lexical analyzer library for JCE language in NodeJS

Usage

var lexer = require("node-jce-lexer");
var tokenStream = lexer.lexUnit.tokenize(codeText);
/* Now do what you want with token stream */

Token stream is actually an array of tokens. Single token is a single javascript object. Format of a single token is following

{
    "lexeme": "module",
    "row": 5,
    "col": 3,
    "tokenClass": "MODULE",
    "keyword": true,
    "parent": null,
    "child": null
}

parent and child these two are kept so that parse tree can be built using the tokens as nodes.