css-tokens
A regex that tokenizes CSS.
A regex that tokenizes CSS.
Easily replace and transform :props in strings.
POS Tagger and lemmatizer for javascript
A simple, Twitter-aware tokenizer.
Developer friendly Natural Language Processing ✨
streaming html tokenizer
Splits a string into tokens by a given separator, treating any quoted parts as a single token.
This module covers some basic nlp principles and implementations. Every implementation in this module is written as stream to only hold that data in memory that is currently processed at any step.
A simple AST generator.
A simple iterative lexer written in TypeScript
Tokenizer that transforms a string of sentences into an array of white-space separated strings of tokens
tiny tokenizer for simple parsing
String Tokenizer for Node.js using ICU's BreakIterators
String ngram splitter.
Replaces spots in a string with their values in an object. Super simple templating engine.
Uses esprima to extract line and block comments from a string of JavaScript. Also optionally parses code context (the next line of code after a comment).
Tokenize a string with regex.
Splits a JSON string into an annotated list of tokens
A gemtext (`text/gemini`) parser with support for streaming, ASTs, and CSTs
Easy way to split words