@kessler/tokenize
personal tokenize util
personal tokenize util
printf format string tokenize
Pacote desenvolvido para ajudar a criar tokens rapidamente.
Convenient lexical scanner inspired by Ruby StringScanner
JavaScript library for tokenizing strings with regular expressions
A CLI tool for converting JSX files into standard JS.
A CLI command for converting JSX files into standard JS.
Tokenizer for js, jsx, (ts, tsx)
convert a string to list of words
hast utility to parse from HTML
Lemmize and tokenize string which contains Chinese and English words
json-operations allows you to work with JSON, parse in a standardized way, get tokenize JSON, etc...
A regex that tokenizes JavaScript.
html tokenizer
micromark utility to tokenize subtokens
NativeExtractor bindings for Node.js
Transform hypertext strings (e.g., HTML, Markdown) into plain text for natural language processing (NLP) normalization
Simple, but powerful lexical scanner that is a more minimal implementation of X-Scanner
Convenient lexical scanner inspired by Ruby's StringScanner
get word counts / frequencies on a per-speaker or per-category basis, or as an aggregate