Cjs-module-lexer-rs NPM

NPM

CJS Module Lexer (Rust)

This is a rewrite of cjs-module-lexer in Rust. It is a CommonJS lexer used to detect the most likely list of named exports of a CommonJS module.

Online Playground

CJS Module Lexer Playground

Installation

WebAssembly JS Wrapper

With Package Managers

npm i cjs-module-lexer-rs

yarn add cjs-module-lexer-rs

pnpm add cjs-module-lexer-rs

With esm.sh

import { init, parse } from "https://esm.sh/cjs-module-lexer-rs";

Rust

Coming soon...

Get Started

Node

// example.js
const { init, parse } = require("cjs-module-lexer-rs");

const code = `
    module.exports.asdf = 'asdf';
    exports = 'asdf';
    module.exports = require('./asdf');
    if (maybe)
    module.exports = require("./another");
`;

init().then(() => console.log(parse(code, 'filename.js')));

{
  imports: [ './asdf', './another'],
  exports: [ 'asdf' ],
  reexports: [ './another' ],
  errors: []
}

Web

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <title>Lexer Example</title>
    </head>
    <body>
        <script type="module">
            import { init, parse } from "https://esm.sh/cjs-module-lexer-rs";

            const code = `
                module.exports.asdf = 'asdf';
                exports = 'asdf';
                module.exports = require('./asdf');
                if (maybe)
                module.exports = require("./another");
            `;

            init().then(() => console.log(parse(code, 'filename.js')));
        </script>
    </body>
</html>

{
    "imports": [
        "./asdf",
        "./another",
    ],
    "exports": [
        "asdf"
    ],
    "reexports": [
        "./another"
    ],
    "errors": []
}

The Why

The frontend tooling has migrated to usage of moderm native languages to accelerate development and improve developer experience. cjs-module-lexer remains useful in many scenes, but it assumes the code is UTF16 and only allows single-thread. Making cjs-module-lexer working with Rust requires FFI and unsafe code. Hopefully this Rust library will be useful for Rust tooling authors to interop between CJS and ESM.

Features

Feature	Status	Since	Note
`exports.asdf = x`	👌	0.1.0
`exports['asdf'] = x`	👌	0.1.0
`module.exports = { ... }`	👌	0.1.0	`{ ... }` is like `{ a, b, c: d }`, where `d` is Literal or Identifier
`require('module')`	👌	0.1.0
`Object.defineProperty(exports, 'q', { enumerable: true, get() { return q } })`	👷		TypeScript: `export {colorFactory} from './color-factory';`
`__export`, `__exportStar`	👷		TypeScript: `export * from 'external'`
Skip StringLiteral	👌	0.1.0
Skip RegularExpressionLiteral	👌	0.1.0
Skip Template	👌	0.1.0
Non-unicode Named Export	❌		Not supported due to `std::str` only allows unicode strings

Reference

https://babeljs.io/docs/en/babel-plugin-transform-modules-commonjs

Benchmarks

Native

cargo bench

test tests::bench_angular          ... bench:   5,176,437 ns/iter (+/- 260,598)
test tests::bench_angular_min      ... bench:   2,321,468 ns/iter (+/- 939,693)
test tests::bench_d3               ... bench:   3,155,154 ns/iter (+/- 182,490)
test tests::bench_d3_min           ... bench:   1,897,146 ns/iter (+/- 100,576)
test tests::bench_magic_string     ... bench:     209,630 ns/iter (+/- 11,577)
test tests::bench_magic_string_min ... bench:     145,834 ns/iter (+/- 7,527)
test tests::bench_rollup           ... bench:   3,918,989 ns/iter (+/- 130,726)
test tests::bench_rollup_min       ... bench:   2,637,478 ns/iter (+/- 74,449)

@everything-registry/sub-chunk-1329 @infinitebrahmanuniverse/nolb-cj

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago