Tokenize | npm.io

Tiny JavaScript tokenizer.

JavaScript js ECMAScript es token tokens tokenize tokenizer regex regexp

9.0.0 • Published 2 months ago

Transform stream that tokenizes CSS

css tokenize parse stream streaming

1.0.1 • Published 9 years ago

transform stream to tokenize html

html parser tokenize stream

2.0.1 • Published 4 years ago

safely inject a block of tokens into a shader

glsl glslify stackgl token tokenize tokenizer start varying define inject

1.1.0 • Published 8 years ago

lexing json

tokens tokenize whitespace json parse parser

1.2.0 • Published 2 years ago

Fork: HTML parsing/serialization toolset for Node.js. WHATWG HTML Living Standard (aka HTML5)-compliant. Fork from the HTMLParseErrorWG branch.

html parser html5 WHATWG specification fast html parser html5 parser htmlparser parse5

4.0.4 • Published 4 years ago

An elegant armor-plated JavaScript lexer modelled after flex. Easily extensible to tailor to your need for perfection.

lex lexer lexical analysis scan scanner scanning token tokenize tokenizer

1.7.9 • Published 9 years ago

Uses esprima to extract line and block comments from a string of JavaScript. Also optionally parses code context (the next line of code after a comment).

ast block code comment comments context esprima extract glob javascript

1.1.0 • Published 5 years ago

Uses snapdragon to tokenize a single JavaScript block comment into an object, with description, tags, and code example sections that can be passed to any other comment parsers for further parsing.

code comment examples gfm indented javadoc javascript jsdoc parse tokenize

3.0.1 • Published 5 years ago

Converts a string into an array of tokens, with useful methods for looking ahead and behind, capturing, matching, et cetera.

compile compiler convert lexer parse parser render scan scanner snapdragon

4.0.0 • Published 5 years ago

Tokenize a string into words and whitespace tokens

token whitespace tokenize string str words space

0.0.1 • Published 8 years ago

A fast tokenizer/lexer core that is well tested.

tokenizer tokeniser tokenize tokenise lexer token lex parser fast

0.2.2 • Published 3 years ago

Tokenize JavaScript

complexion javascript tokenizer tokeniser tokenize tokenise lexer token lex parser

0.2.2 • Published 3 years ago

WHATWG HTML5 specification-compliant, fast and ready for production HTML parsing/serialization toolset for Node.js

html parser html5 WHATWG specification fast html parser html5 parser htmlparser parse5

2.2.6 • Published 7 years ago

Tokenize Excel formulas

excel formula parse tokenize

3.0.0 • Published 2 years ago

Create a string representation of an object or primative

token tokenize tokenizer toString toSource stringify serialize

0.1.0 • Published 9 years ago

Plugin which allows your users to select multiple items from a predefined list or ajax, using autocompletion as they type to find each item.

tokenizer tokeniser tokenize tokenise tags taggify keywords

1.3.5 • Published 3 years ago

Small, fast, event-driven, fault-tolerant html tokenizer. Works in node or browsers.

html tokenize tokenizer tokenization parser parsing

4.0.1 • Published 3 years ago

Injects and enforces row limits in your SQL statements.

sql tokenize lexer

3.0.0 • Published 2 years ago

Straightforward HTML parser for Node.js and browser

html parser html5 html5 parser htmlparser html parser html tree-constructor html to JSON html to AST html tokenizer

1.1.9 • Published 4 years ago