Codsen-tokenizer NPM | npm.io

7.0.25 • Published 6 months ago

Install

Weekly downloads

92

License

MIT

Repository

Last release

6 months ago

Install

This package is pure ESM. If you're not ready yet, install an older version of this program, 5.6.0 (npm i codsen-tokenizer@5.6.0).

npm i codsen-tokenizer

Quick Take

import { strict as assert } from "assert";

import { tokenizer } from "codsen-tokenizer";

const gathered = [];

// it operates from a callback, like Array.prototype.forEach()
tokenizer("<td nowrap>", {
  tagCb: (obj) => {
    gathered.push(obj);
  },
});

assert.deepEqual(gathered, [
  {
    type: "tag",
    start: 0,
    end: 11,
    value: "<td nowrap>",
    tagNameStartsAt: 1,
    tagNameEndsAt: 3,
    tagName: "td",
    recognised: true,
    closing: false,
    void: false,
    pureHTML: true,
    kind: null,
    attribs: [
      {
        attribName: "nowrap",
        attribNameRecognised: true,
        attribNameStartsAt: 4,
        attribNameEndsAt: 10,
        attribOpeningQuoteAt: null,
        attribClosingQuoteAt: null,
        attribValueRaw: null,
        attribValue: [],
        attribValueStartsAt: null,
        attribValueEndsAt: null,
        attribStarts: 4,
        attribEnds: 10,
        attribLeft: 2,
      },
    ],
  },
]);

Documentation

Please visit codsen.com for a full description of the API.

Contributing

To report bugs or request features or assistance, raise an issue on GitHub.

Licence

MIT License.

Copyright © 2010-2024 Roy Revelt and other contributors.

ast codsen css html lexer lint parse parsing processing token tokeniser tokenizer

codsen-utils html-all-known-attributes is-char-suitable-for-html-attr-name is-html-attribute-closing is-html-tag-opening rfdc string-left-right string-match-left-right

@everything-registry/sub-chunk-1351 stristri codsen-parser

6 months ago

8 months ago

10 months ago

11 months ago

11 months ago

1 year ago

1 year ago

1 year ago

1 year ago

1 year ago

2 years ago

2 years ago

1 year ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

2 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

3 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

4 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago

5 years ago