1.0.11 • Published 4 months ago

@thi.ng/sexpr v1.0.11

Weekly downloads
7
License
Apache-2.0
Repository
github
Last release
4 months ago

sexpr

npm version npm downloads Twitter Follow

This project is part of the @thi.ng/umbrella monorepo.

About

Basic, but configurable and extensible S-Expression tokenizer, parser, AST builder and runtime / interpreter skeleton for custom, sandboxed DSL implementations.

The following default syntax rules are used:

  • whitespace: space, tab, newline, comma
  • expression delimiters: (, )
  • numbers: any float notation valid in JS, hex ints prefixed w/ 0x
  • string delimiters: "

Everything else is parsed as is, i.e. as symbol.

Status

ALPHA - bleeding edge / work-in-progress

Search or submit any issues for this package

Related packages

  • @thi.ng/parse - Purely functional parser combinators & AST generation for generic inputs

Installation

yarn add @thi.ng/sexpr

ES module import:

<script type="module" src="https://cdn.skypack.dev/@thi.ng/sexpr"></script>

Skypack documentation

For Node.js REPL:

# with flag only for < v16
node --experimental-repl-await

> const sexpr = await import("@thi.ng/sexpr");

Package sizes (gzipped, pre-treeshake): ESM: 818 bytes

Dependencies

Usage examples

Several demos in this repo's /examples directory are using this package.

A selection:

ScreenshotDescriptionLive demoSource
rstream based spreadsheet w/ S-expression formula DSLDemoSource

API

Generated API docs

Tokenize only (iterator)

The tokenize function returns an iterator of tokens incl. location details. Any whitespace is skipped and whitespace characters are configurable.

[...tokenize(`(* (+ 3 5) 10)`)];
// [
//   { value: '(', line: 0, col: 0 },
//   { value: '*', line: 0, col: 1 },
//   { value: '(', line: 0, col: 3 },
//   { value: '+', line: 0, col: 4 },
//   { value: '3', line: 0, col: 6 },
//   { value: '5', line: 0, col: 8 },
//   { value: ')', line: 0, col: 9 },
//   { value: '10', line: 0, col: 11 },
//   { value: ')', line: 0, col: 13 }
// ]

AST generation

The parse function takes a source string or iterable of tokens and parses it into an AST.

parse(tokenize(`(* (+ 3 5) 10)`));
// or directly from string
parse(`(* (+ 3 5) 10)`);
{
  "type": "root",
  "children": [
    {
      "type": "expr",
      "value": "(",
      "children": [
        {
          "type": "sym",
          "value": "*"
        },
        {
          "type": "expr",
          "value": "(",
          "children": [
            {
              "type": "sym",
              "value": "+"
            },
            {
              "type": "num",
              "value": 3
            },
            {
              "type": "num",
              "value": 5
            }
          ]
        },
        {
          "type": "num",
          "value": 10
        }
      ]
    }
  ]
}

Interpreter

import { Fn2 } from "@thi.ng/api";
import { defmulti, DEFAULT } from "@thi.ng/defmulti";
import { ASTNode, Implementations, Sym } from "@thi.ng/sexpr";

// multi-dispatch fn for DSL builtins
// we will call this function for each S-expression
// and will delegate to the actual implementation based on
// the expression's first item (a symbol name)
const builtins = defmulti<Sym, ASTNode[], any>((x) => x.value);

// build runtime w/ impls for all AST node types
// the generics are the types of: the custom environment (if used)
// and the result type(s)
const rt = runtime<Implementations<any,any>, any, any>({
    // delegate to builtins
    expr: (x, env) => builtins(<Sym>x.children[0], x.children, env),
    // lookup symbol in environment
    sym: (x, env) => env[x.value],
    // strings and numbers evaluate verbatim
    str: (x) => x.value,
    num: (x) => x.value
});

// helper HOF for math ops
const op = (fn: Fn2<number, number, number>) =>
    (_: ASTNode, vals: ASTNode[], env: any) =>
        vals.slice(2).reduce(
            (acc, x) => fn(acc, rt(x, env)),
            rt(vals[1], env)
        );

// add builtins
builtins.addAll({
    "+": op((acc, x) => acc + x),
    "*": op((acc, x) => acc * x),
    "-": op((acc, x) => acc - x),
    "/": op((acc, x) => acc / x),
    count: (_, [__, x]) => rt(x).length
});

// add default/fallback implementation
// to allow calling functions stored in environment
builtins.add(DEFAULT, (x, [_, ...args], env) => {
    const f = env[(<Sym>x).value];
    assert(!!f, "missing impl");
    return f.apply(null, args.map((a) => rt(a, env)));
});

// evaluator
const $eval = (src: string, env: any = {}) =>
    rt(parse(src).children[0], env);

// evaluate expression w/ given env bindings
$eval(`(* foo (+ 1 2 3 (count "abcd")))`, { foo: 10 });
// 100
// i.e. 100 = 10 * (1 + 2 + 3 + 4)

// call env function
$eval(
    `(join (+ 1 2) (* 3 4))`,
    { join: (...xs: any[]) => xs.join(",") }
);
// "3,12"

See test/ for a more in-depth version of this example...

Custom syntax

// define syntax overrides (keep default whitespace rules)
const syntax = {
    scopes: [["<", ">"], ["{", "}"]],
    string: "'"
};

parse(`<nest { a '2' b 3 }>`, syntax);
{
  "type": "root",
  "children": [
    {
      "type": "expr",
      "value": "<",
      "children": [
        {
          "type": "sym",
          "value": "nest"
        },
        {
          "type": "expr",
          "value": "{",
          "children": [
            {
              "type": "sym",
              "value": "a"
            },
            {
              "type": "str",
              "value": "2"
            },
            {
              "type": "sym",
              "value": "b"
            },
            {
              "type": "num",
              "value": 3
            }
          ]
        }
      ]
    }
  ]
}

Authors

Karsten Schmidt

If this project contributes to an academic publication, please cite it as:

@misc{thing-sexpr,
  title = "@thi.ng/sexpr",
  author = "Karsten Schmidt",
  note = "https://thi.ng/sexpr",
  year = 2019
}

License

© 2019 - 2021 Karsten Schmidt // Apache Software License 2.0

1.0.11

4 months ago

1.0.10

5 months ago

1.0.9

5 months ago

1.0.8

5 months ago

1.0.7

5 months ago

1.0.6

5 months ago

1.0.5

6 months ago

1.0.4

6 months ago

1.0.3

6 months ago

1.0.2

6 months ago

1.0.1

6 months ago

0.5.54

7 months ago

0.5.53

7 months ago

0.5.52

8 months ago

0.5.51

8 months ago

0.5.50

9 months ago

0.5.43

1 year ago

0.5.44

12 months ago

0.5.41

1 year ago

0.5.42

1 year ago

0.5.49

10 months ago

0.5.47

11 months ago

0.5.48

11 months ago

0.5.45

12 months ago

0.5.46

12 months ago

0.5.40

1 year ago

0.5.39

1 year ago

0.5.38

1 year ago

0.5.37

1 year ago

0.5.36

1 year ago

0.5.35

1 year ago

0.5.34

1 year ago

0.5.33

1 year ago

0.5.32

1 year ago

0.5.31

1 year ago

0.5.30

1 year ago

0.5.29

1 year ago

0.5.28

1 year ago

0.5.27

1 year ago

0.5.25

1 year ago

0.5.26

1 year ago

0.5.24

1 year ago

0.5.23

1 year ago

0.5.22

1 year ago

0.5.21

1 year ago

0.5.18

1 year ago

0.5.19

1 year ago

0.5.17

1 year ago

0.5.20

1 year ago

0.5.16

1 year ago

0.5.15

1 year ago

0.5.14

2 years ago

0.5.13

2 years ago

0.5.11

2 years ago

0.5.12

2 years ago

0.5.10

2 years ago

0.5.9

2 years ago

0.4.43

2 years ago

0.4.41

2 years ago

0.4.46

2 years ago

0.4.47

2 years ago

0.4.44

2 years ago

0.4.45

2 years ago

0.5.8

2 years ago

0.5.7

2 years ago

0.5.4

2 years ago

0.5.3

2 years ago

0.5.6

2 years ago

0.5.0

2 years ago

0.5.2

2 years ago

0.5.1

2 years ago

0.4.40

2 years ago

0.4.39

2 years ago

0.4.38

2 years ago

0.4.37

2 years ago

0.4.35

2 years ago

0.4.36

2 years ago

0.4.34

2 years ago

0.4.33

2 years ago

0.4.32

2 years ago

0.4.30

2 years ago

0.4.28

3 years ago

0.4.29

3 years ago

0.4.26

3 years ago

0.4.27

3 years ago

0.4.20

3 years ago

0.4.21

3 years ago

0.4.24

3 years ago

0.4.25

3 years ago

0.4.22

3 years ago

0.4.23

3 years ago

0.4.19

3 years ago

0.4.17

3 years ago

0.4.18

3 years ago

0.4.16

3 years ago

0.4.15

3 years ago

0.4.14

3 years ago

0.4.9

3 years ago

0.4.10

3 years ago

0.4.13

3 years ago

0.4.11

3 years ago

0.4.12

3 years ago

0.4.8

3 years ago

0.4.7

3 years ago

0.4.6

3 years ago

0.4.5

3 years ago

0.4.4

3 years ago

0.3.8

4 years ago

0.4.1

4 years ago

0.4.0

4 years ago

0.4.3

4 years ago

0.4.2

4 years ago

0.3.7

4 years ago

0.3.6

4 years ago

0.3.4

4 years ago

0.3.0

4 years ago

0.3.1

4 years ago

0.3.3

4 years ago

0.2.48

4 years ago

0.2.47

4 years ago

0.2.46

4 years ago

0.2.45

4 years ago

0.2.44

4 years ago

0.2.43

4 years ago

0.2.42

4 years ago

0.2.41

4 years ago

0.2.40

4 years ago

0.2.39

4 years ago

0.2.38

4 years ago

0.2.36

4 years ago

0.2.34

4 years ago

0.2.33

5 years ago

0.2.32

5 years ago

0.2.31

5 years ago

0.2.30

5 years ago

0.2.29

5 years ago

0.2.28

5 years ago

0.2.27

5 years ago

0.2.26

5 years ago

0.2.25

5 years ago

0.2.24

5 years ago

0.2.23

5 years ago

0.2.22

5 years ago

0.2.21

5 years ago

0.2.20

5 years ago

0.2.19

5 years ago

0.2.18

5 years ago

0.2.17

5 years ago

0.2.16

5 years ago

0.2.15

5 years ago

0.2.14

5 years ago

0.2.13

5 years ago

0.2.12

5 years ago

0.2.11

5 years ago

0.2.10

5 years ago

0.2.9

5 years ago

0.2.8

5 years ago

0.2.7

5 years ago

0.2.6

5 years ago

0.2.4

5 years ago

0.2.3

5 years ago

0.2.2

6 years ago

0.2.1

6 years ago

0.2.0

6 years ago

0.1.0

6 years ago