Tokenize | npm.io

Simple synchronous string tokenizer using Regex

tokenizer parser compiler tokenize

2.0.0 • Published 8 years ago

Tokenize Excel formulas

excel formula parse tokenize

2.3.2 • Published 5 years ago

Wrapper of monaco-editor to use some functionalities in Node.js.

monaco editor vscode node synthax code ast highlight text tokenize

0.3.0 • Published 3 years ago

Used to interact with the OpenToken API.

tokenize opentoken library lib

1.0.1 • Published 8 years ago

Basic English tokenizer

token tokenize tokenization words

1.0.6 • Published 9 years ago

Parse regular expressions into an AST.

ast tokenize tokens parse parse-regex parser regex regular expression

0.1.0-beta • Published 7 years ago

Module for fast element to selector matching (with custom :hover, :focus etc. handing)

css token tokenize tokenizer string selector grammar parser lexer spec

0.9.2 • Published 9 years ago

Module for fast css selector tokenization from string

css token tokenize tokenizer string selector grammar parser lexer spec

0.9.5 • Published 9 years ago

string/array poking for pet params

utils poke token tokenize string array params args argv

0.6.11 • Published 11 years ago

Binary wrapper for Sedlex_ppx - A lexer generator for OCaml, similar to ocamllex, but supporting Unicode.

BuckleScript ReasonML OCaml CLI bin binary PPX AST transpiler tool

20.0.0 • Published 6 years ago

es6 version of aaditmshah/lexer

lex es6 es2015 lexer lexical analysis scan scanner scanning token

1.1.0 • Published 10 years ago

Time a JavaScript tokenizer

complexion tokenizer tokeniser tokenize tokenise profile

1.1.0 • Published 8 years ago

The most standards compliant CSS parser.

css parse tokenize syntax

0.1.0 • Published 11 years ago

A streaming JSON tokenizer

stream streaming json tokenizer tokenize

1.1.0 • Published 8 years ago

tokenize sort string

sort tokenize token

1.0.4 • Published 7 years ago

A HTML tokenizer

html tokenize

1.0.2 • Published 9 years ago

Read a file, tokenize it, and spit out a handy JSON.

tokenize nlp

0.1.7 • Published 7 years ago

transform stream to tokenize mustache

stache tokenize parse mustache html

0.1.0 • Published 10 years ago

Tokenizes utterances that contain a mix of English and Chinese words.

tokenizer chinese english language tokenize nltk nlp natural-language-processing chinese-tokenizer contains-chinese

1.1.0 • Published 7 years ago

HTML parsing/serialization toolset for Node.js. WHATWG HTML Living Standard (aka HTML5)-compliant.

html parser html5 WHATWG specification fast html parser html5 parser htmlparser parse5

3.0.3 • Published 8 years ago

13 4 57 8

Tokenize Packages

tokenize-sync

@modelmap/excel-formula-tokenizer

monaco-editor-node

opentoken-lib

ogden

parse-regex

selector-checker

selector-tokenizer

poketh

ppx-sedlex

lex-es6

complexion-time-map

cssprima

json-tokenizer

token-sort

tokenizer-of-html

tokenize-file

tokenize-stache

mixed-english-and-chinese-tokenizer

parse5-wxmp