# complexion-js

> Tokenize JavaScript

Latest version **0.2.2** (published 2021-05-19) · MIT license · 0 weekly downloads

## Install

```sh
npm install complexion-js
pnpm add complexion-js
yarn add complexion-js
bun add complexion-js
```

## Health

**Score 25/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support; pre 1.0.

Negative: abandoned.

## Facts

| | |
|---|---|
| Version | 0.2.2 |
| Published | 2021-05-19 |
| First published | 2014-06-07 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 0 |
| Unpacked size | 73.6 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 2 |
| Author | Tyler Akins |
| Maintainers | fidian |
| Keywords | complexion, javascript, tokenizer, tokeniser, tokenize, tokenise, lexer, token, lex, parser, fast |

## Links

- npm: https://www.npmjs.com/package/complexion-js
- Repository: https://github.com/tests-always-included/complexion-js
- Issues: https://github.com/tests-always-included/complexion-js/issues
- npm.io page: https://npm.io/package/complexion-js

## Alternatives

- [babylon](https://npm.io/package/babylon.md) — 5.1M weekly downloads
- [csscolorparser](https://npm.io/package/csscolorparser.md) — 3.7M weekly downloads
- [expr-eval-fork](https://npm.io/package/expr-eval-fork.md) — 1.5M weekly downloads
- [@leeoniya/ufuzzy](https://npm.io/package/@leeoniya/ufuzzy.md) — 247.7K weekly downloads
- [xml-parser](https://npm.io/package/xml-parser.md) — 78.4K weekly downloads

## Recent versions

- 0.2.2 (latest) — 2021-05-19
- 0.2.1 — 2019-11-20
- 0.2.0 — 2017-07-25
- 0.1.5 — 2015-01-13
- 0.1.4 — 2014-11-21
- 0.1.3 — 2014-10-12
- 0.1.2 — 2014-10-12
- 0.1.1 — 2014-06-07
- 0.1.0 — 2014-06-07

## README

Complexion - JavaScript
=======================

Extends the [Complexion] tokenizing library so it is able to parse JavaScript into tokens.  Useful if you want to create a JavaScript pretty printer, parser or anything that would need to understand the syntax of JavaScript.

[![npm version][npm-badge]][npm-link]
[![Build Status][travis-badge]][travis-link]
[![Dependencies][dependencies-badge]][dependencies-link]
[![Dev Dependencies][devdependencies-badge]][devdependencies-link]


Usage
-----

Install this package and [Complexion] with `npm` or `bower` or maybe just clone the two repositories.  Then, use the library in your JavaScript.  There is UMD markup (via [FidUmd]) that should make this library available under any module system that you prefer to use.

    // NPM
    var Complexion = require('complexion');
    var instance = new Complexion();
    require('complexion-js')(instance);
    var tokenList = instance.tokenize('... JavaScript goes here ...');

    // Browser
    var instance = new Complexion();
    complexionJs(instance);
    var tokenList = instance.tokenize('... JavaScript goes here ...');

Options may be passed as well.

    complexionJs(instance, {
        bom: false,
        shebang: false
    });

Options
-------

* bom (boolean, enabled by default) - Allow searching for the byte order mark in files.
* shebang (boolean, enabled by default) - Allow searching for a shebang at the top of a file.

Token Types
-----------

The following are all of the available tokens that the tokenizer produces.

* `BOM` - UTF8 byte order mark at the very beginning of the file.
* `BOOLEAN_LITERAL` - Either `true` or `false`.
* `IDENTIFIER_NAME` - The names of functions, variables, properties.  This does not include keywords.
* `IMPLICIT_SEMICOLON` - A special token that has no content but indicates that there should have been a semicolon here.  It's necessary for determining context and if something is division or a regular expression.
* `KEYWORD` - Special words that the language reserves and are not allowed as variable names.  Does not include null nor booleans.
* `LINE_TERMINATOR` - Newlines (`\r\n`, `\n` and `\r`).
* `MULTI_LINE_COMMENT` - A comment that uses `/* */`.
* `NULL_LITERAL` - Only matches `null`.
* `NUMERIC_LITERAL` - A number in decimal, octal or hex.  Decimals may have exponents.
* `PUNCTUATOR` - Most symbols including math, array literals, object literals, function calls.
* `REGULAR_EXPRESSION_LITERAL` - Regular expressions can be tricky because they sometimes look like division and have lots of allowed escape sequences.
* `SHEBANG` - Only matches the first line when it starts with `#!` and can be turned off.
* `SINGLE_LINE_COMMENT` - A comment that starts with `//`.
* `STRING_LITERAL` - A single or double quoted string.
* `UNKNOWN` - An unknown character.  If you hit this, you have poorly formed JavaScript.  This is not a full lint test, just the beginning of one.
* `WHITESPACE` - Spaces and tabs.  Not newlines.


Token Objects
-------------

The tokens returned from tokenization will all be instances of a `ComplexionJsToken` class.  They retain the same properties that the original token object has and get additional methods.


### ComplexionJsToken.prototype.isAnyType(tokenTypes)

Returns `true` if the token is any of the types passed in.

    if (token.isAnyType([ 'WHITESPACE', 'LINE_TERMINATOR' ])) {
        // This token does not appear on screens
    }


### ComplexionJsToken.prototype.isAnyType(tokenTypes)

Returns `true` if the token is exactly the type passed in.

    if (token.isType('BOOLEAN_LITERAL')) {
        // token.content === "true" || token.content === "false"
    }


### ComplexionJsToken.prototype.isAnyType(tokenTypes)

Returns `true` when the token could be eliminated entirely from the output.

    // Simple minifier
    tokenList.forEach(function (token) {
        if (!token.isUnimportant()) {
            if (token.isType('IMPLICIT_SEMICOLON')) {
                console.log(';');
            } else {
                console.log(token.content);
            }
        }
    });


Development
-----------

If you want to work on this library, you need to check out the repository and run `npm install` to get the dependencies.

Tests are *always* included.  Make sure tests cover your changes.  To run the current tests, just use `npm test` or `grunt test` (they will run the same test suite).

Make sure the matchers are fast.  This library is tweaked to work on unminified source, so take that into consideration before shuffling the matchers too much.


License
-------

This software is licensed under an [MIT license with an additional non-advertising clause](LICENSE.md).


[Complexion]: https://github.com/tests-always-included/complexion
[dependencies-badge]: https://img.shields.io/david/tests-always-included/complexion-js.svg
[dependencies-link]: https://david-dm.org/tests-always-included/complexion-js
[devdependencies-badge]: https://img.shields.io/david/dev/tests-always-included/complexion-js.svg
[devdependencies-link]: https://david-dm.org/tests-always-included/complexion-js#info=devDependencies
[FidUmd]: https://github.com/fidian/fid-umd/
[LICENSE]: LICENSE.md
[npm-badge]: https://img.shields.io/npm/v/complexion-js.svg
[npm-link]: https://npmjs.org/package/complexion-js
[travis-badge]: https://img.shields.io/travis/tests-always-included/complexion-js/master.svg
[travis-link]: http://travis-ci.org/tests-always-included/complexion-js

---
_Source: https://npm.io/package/complexion-js · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._