s-expression.js v0.6.4
S-Expression Parser, Serializer, Interpreter, and Tree Constructor / Walker Utilities in plain JavaScript for browsers and Node.js with zero dependencies.
Feature Highlights
Parse S-Expression string to Abstract Syntax Tree (AST)
- AST data structure is JSON (ideal for program/data transfer or interpretation).
Construct and traverse AST using helper methods
Serialize AST to S-Expression string
Support LISP-like comment syntax: line comment prefix
;
and block comment guards#|
,|#
- Comments are stripped in the parsing process and therefore not included in AST.
Support JSON-compatible value types:
boolean
,number
,null
, andstring
with multi-line support and escaped quotation\"
Interpret AST as LISP-like functional notation (a.k.a Cambridge Polish Notation), convenient for making Domain Specific Languages (DSLs) that have similar syntax to LISP dialects (CLIPS, Clojure, Scheme, Racket, etc.).
Define your custom handlers in JS to evaluate expressions in a similar concept like in YACC/Bison or ANTLR but without the out-of-source file for code generation. With S-Expression.js, the minimum grammar is presumed to be LISP-like, so the parsing is taken care of, and all you just need to do is to define the expression evaluators which can be done all in-source.
The default transformed format is also JSON, so if you like, you can defer to another language for the evaluation part. Since this library is self-contained, and because JavaScript is widely supported, you can easily interop it with a sandbox in your application such that you retrieve just the JSON output to evaluate using features available in your host language.
JSON
array
andobject
types are used to represent function forms.
TL;DR: jump to Quick Start and hack away
โญ Background
๐ก Why make domain-specific languages?
All usual reasons for DSLs apply. Here are just a few use cases:
Custom data format that is more compact and easier to write than the target format schema
Custom command format for system interaction or utility (e.g. REPL shell, chat bot command, etc.)
Abstract away low-level/repetitive JS code so that developers/users can write in a high-level language for the business logic / query.
Allow clients to send a remote procedure in a safe DSL so that the server can interpret just what is allowed.
Alternative to adding programming logic to a dumb data format like JSON/YAML/XML when it started as a quick workaround at first, but then the logic grows.
๐ ๏ธ Installation
Option 1: install latest release on NPM to your Node.js project
npm install --save s-expression.js
Option 2: install from this repository source code
git clone https://github.com/NLKNguyen/S-Expression.JS
cd S-Expression.JS
# make it globally available on your machine; to undo: npm unlink
npm link
# then from your own project directory, make a link to this library; similarly to undo: npm unlink s-expression.js
npm link s-expression.js
# you can now require it in your JS source just like normal, e.g. const SExpr = require("s-expression.js")
๐งช Test
Run all test cases
npm run test
Run a specific set of test cases
npx tape ./tests/parse.js | npx tap-spec
npx tape ./tests/interpret.js | npx tap-spec
npx tape ./tests/helpers.js | npx tap-spec
Piping to tap-spec
is optional, but it makes the output easier to read.
๐ Quick start
API documentation is embedded at the bottom for reference.
parse(str: string): Array< number | string | Array >
Turns a raw S-expression string into JSON that represents an abstract syntax tree in which:
- An expression is parsed as a JSON array
- A number is parsed as a JSON number
- A double-quoted string is parsed as a JSON string wrapped with literal double quotations, e.g.
"\"example string\""
S.valueOf("\"example string\"")
returns"example string"
JSON string value
- A boolean (
true
,false
,#t
,#f
) is parsed as a JSON string of the value, e.g."true"
S.valueOf("true")
returnstrue
JSON boolean value- Tokens that indicate boolean values can be configured
- A null (
null
,#nil
) is parsed as a JSON string of the null value, e.g."null"
S.valueOf("null")
returnsnull
JSON value- Tokens that indicate null can be configured
- Note:
undefined
is not configured, so it's parsed as a normal atom.
- Note:
- Any other token is parsed as a JSON string of the identifier, e.g.
"a"
S.valueOf("a")
returns"a"
JSON string value
If the return is not an array (can check with S.isExpression
), then it's invalid.
- Initialize
const SExpr = require("s-expression.js")
const S = new SExpr()
- Simple parsing root S-expression as a list
S.parse(`(1 2 3)`) // [1, 2, 3]
- Simple tree-walk on root expression with mixed value types
const ast = S.parse(`( 1 "a \\"b\\" c" true null d (e f ()) )`)
if (S.isExpression(ast)) {
console.log(`ast is an expression: ${JSON.stringify(ast)}`)
} else {
throw Error(`ast is not a valid expression`)
}
let index = 0
for (let e of ast) {
if (S.isNumber(e)) {
console.log(`ast[${index}] is a number with value: ${S.valueOf(e)}`)
} else if (S.isString(e)) {
console.log(`ast[${index}] is a string with value: ${JSON.stringify(S.valueOf(e))}`)
} else if (S.isBoolean(e)) {
console.log(`ast[${index}] is a boolean with value: ${S.valueOf(e)}`)
} else if (S.isNull(e)) {
console.log(`ast[${index}] is a null with value: ${S.valueOf(e)}`)
} else if (S.isAtom(e)) {
console.log(`ast[${index}] is an atom with id: ${S.valueOf(e)}`)
} else { // S.isExpression(e)
console.log(`ast[${index}] is an expression: ${JSON.stringify(e)}`)
}
index++
}
Output:
ast is an expression: [1,"\"a \\\"b\\\" c\"","true","null","d",["e","f",[]]]
ast[0] is a number with value: 1
ast[1] is a string with value: "a \\\"b\\\" c"
ast[2] is a boolean with value: true
ast[3] is null: null
ast[4] is an atom with id: d
ast[5] is an expression: ["e","f",[]]
See ./tests/parse.js
for more parsing examples
Serialize
TODO
Please look at the unit tests for use cases in the meantime.
Interpret
TODO: simple custom DSL example
Please look at the unit tests for use cases in the meantime.
๐ Author
๐ค Nikyle Nguyen
- Website: https://dephony.com/Nikyle
- Twitter: @NLKNguyen
- Github: @NLKNguyen
- LinkedIn: @NLKNguyen
๐ค Contributing
Give a โญ๏ธ if this project helped you working with S-Expression easily in JavaScript!
Contributions, issues and feature requests are welcome! Feel free to check issues page.
๐ Your support is very much appreciated
I create open-source projects on GitHub and continue to develop/maintain them as they are helping others. You can integrate and use these projects in your applications for free! You are free to modify and redistribute any way you like, even in commercial products.
I try to respond to users' feedback and feature requests as much as possible. This takes a lot of time and effort (speaking of mental context-switching between different projects and daily work). Therefore, if these projects help you in your work, and you want to support me to continue developing, here are a few ways you can support me:
- ๐ฌ Following my blog and social profiles listed above to help me connect with your network
- โญ๏ธ Starring this project and sharing with others as more users come, more great ideas arrive!
- โ๏ธ Donating any amount is a great way to help me work on the projects more regularly!
๐ License
Copyright ยฉ 2022 Nikyle Nguyen
The project is MIT License.
It is a simple permissive license with conditions only requiring the preservation of copyright and license notices. Licensed works, modifications, and larger works may be distributed under different terms and without source code.
API
Table of Contents
- SExpr
SExpr
Class of S-Expression resolver that includes parser, serializer, tree constructors, and tree walker utilities.
Creates an instance of SExpr. Optional options
input for configuring
default behavior, such as how to recognize null, boolean values as it's up to
the programmer to decide the syntax. Nevertheless, here is the default that
you can override.
{
truthy: ['true', '#t'],
falsy: ['false', '#f'],
nully: ['null', '#nil']
}
Parameters
options
any (optional, default{}
)
interpret
interpret a parsed expression tree (AST) into data structures in according to a notation type, currently just "functional" notation which is similar to LISP dialects such as CLIPS, Clojure, Scheme, Racket, etc.
Parameters
expression
context
(optional, default{}
)state
(optional, default{scoped:[],globals:{}}
)entity
(optional, defaultthis.ROOT
)E
any
Returns any
stripComments
strip comments from S-expression string
Parameters
str
string code which might have comments
Returns string code without comments
parse
Parse a S-expression string into a JSON object representing an expression tree
Parameters
str
string S-expression stringopts
any deserializing options (optional, default{includedRootParentheses:true}
)
Returns json an expression tree in form of list that can include nested lists similar to the structure of the input S-expression
serialize
Serialize an expression tree into an S-expression string
Parameters
ast
any parsed expression (abstract syntax tree)opts
any serializing options (optional, default{includingRootParentheses:true}
)level
(optional, default0
)
Returns any
identifier
Create an identifier symbol
Parameters
id
string
Examples
const S = new SExpr()
const node = S.expression(S.identifier('a'))
// ['a']
Returns string symbol
isAtom
Check if a node is an identifier, optionally compare to a given name
Parameters
e
any a node to checkid
string optional id name to compare to (optional, defaultundefined
)
Examples
const S = new SExpr()
const node = S.expression(S.identifier('a'))
console.log(S.isAtom(S.first(node)))
// true
console.log(S.isAtom(S.first(node, 'a')))
// true
Returns boolean true if it is an identifier
isEqual
Compare whether 2 nodes are identical
Parameters
a
any a nodeb
any another node to compare to
Returns boolean true if they are the same
expression
Create an expression node
Parameters
exps
rest optional initialization list of elements
Returns json a tree node
isExpression
Check if a node is an expression, and optionally compare to a given expression
Parameters
e
any a node to check whether it's an expressions
json optional expression to compare to (optional, defaultundefined
)
Returns boolean true if it's an expression (and equals the compared expression if provided)
boolean
Create a boolean node with given state
Parameters
v
boolean boolean value
Returns string a node with name corresponding to a boolean value
isBoolean
Check if a node is a boolean value, optionally compare to a given state
Parameters
e
any a node to check whether it's a booleanb
boolean optional state to compare to (optional, defaultundefined
)
Returns boolean true if it's a boolean (and equals the given state if provided)
isTruthy
Check if a node is considered truthy. Anything but an explicit false value is truthy.
Parameters
e
any a node to check if it's truthy
Returns boolean true if it's truthy
isMissing
Check if a node doesn't exist, a.k.a undefined
Parameters
e
any a node to check if it doesn't exist
Returns boolean true if it doesn't exist (undefined)
null
Create a null node.
Returns string a node with name representing null value
isNull
Check if a node is null.
Parameters
e
any a node to check if it's null
Returns boolean true if it's null
number
Create a number node
Parameters
n
number value of the new node
Returns number a node with number value
isNumber
Check if a node is a number
Parameters
e
any a node to check if it's a number, optionally compare to a given valuen
number an optional value to compare to (optional, defaultundefined
)
Returns boolean true if it's a number (and equals the given value if provided)
string
Create a string node.
Parameters
str
string string value of the node
Returns string a node with string value
isString
Check if a node is a string, optionally compare to a given string.
Parameters
e
any a node to check if it's a strings
string optional string to compare to (optional, defaultundefined
)
Returns any true if it's a string (and equals the given string if provided)
valueOf
Get a value content of a symbol (not expression).
Parameters
e
any a node to extract value
Returns any value
first
Get the 1st child of a node.
Parameters
e
any a node to get its child
Returns any a child node if exists
second
Get the 2nd child of a node.
Parameters
e
any a node to get its child
Returns any a child node if exists
third
Get the 3rd child of a node.
Parameters
e
any a node to get its child
Returns any a child node if exists
fourth
Get the 4th child of a node.
Parameters
e
any a node to get its child
Returns any a child node if exists
fifth
Get the 5th child of a node.
Parameters
e
any a node to get its child
Returns any a child node if exists
sixth
Get the 6th child of a node.
Parameters
e
any a node to get its child
Returns any a child node if exists
seventh
Get the 7th child of a node.
Parameters
e
any a node to get its child
Returns any a child node if exists
eighth
Get the 8th child of a node.
Parameters
e
any a node to get its child
Returns any a child node if exists
ninth
Get the 9th child of a node.
Parameters
e
any a node to get its child
Returns any a child node if exists
tenth
Get the 10th child of a node.
Parameters
e
any a node to get its child
Returns any a child node if exists
nth
Get the n-th child of a node. Similar to the shorthand first
, second
, third
, fourth
, fifth
... tenth
, but at any position provided.
Parameters
e
any a node to get its childn
number position of the child node, starting from 1
Returns any a child node if exists
rest
Skip the first child node and get the rest
Parameters
e
any a node to get its child
Returns any the rest of the nodes or undefined if the input node is not an expression