lex
An elegant armor-plated JavaScript lexer modelled after flex. Easily extensible to tailor to your need for perfection.
An elegant armor-plated JavaScript lexer modelled after flex. Easily extensible to tailor to your need for perfection.
JavaScript implementation of Japanese morphological analyzer
Straightforward fuzzy matching, information retrieval and NLP building blocks for JavaScript.
simple code like tokenizer
React-based typeahead and typeahead-tokenizer
A parser generator with a Bison/YACC-similar API (derived off zaach/jison repo)
Node.js SQL highlighter for CLI based on doctrine/sql-formatter.
novel-segment segment data
Chinese word segmentation 簡繁中文分词模块 以網路小說為樣本
tokenize any text stream given some basic regex rules to match tokens
Converts a string into an array of tokens, with useful methods for looking ahead and behind, capturing, matching, et cetera.
Tokenize paragraphs into sentences, and smaller tokens.
Turns a string into tokens.
Returns a parsed representation of a glob string; does not require Minimatch.
A fast tokenizer/lexer core that is well tested.
Tokenize JavaScript
A streaming tokenizer
Input field with tagging/token/chip capabilities written in raw JavaScript
WHATWG HTML5 specification-compliant, fast and ready for production HTML parsing/serialization toolset for Node.js
A regular expression (RexExp) stream tokenizer.