Tokenizer Packages

An object-oriented programming language

React-based typeahead component

tokenize code bricks for JavaScript

A small ECMAScript parser, tokenizer and minifier written in JavaScript.

A JS implementation of a tokenizer to split up source code based on tokens

An experimental virtual machine running on adaptive neuronal networks. Built of an official research-project.

Bluesocks is a lightweight lexer for Javascript

True ITK Text Mining Toolbox for Node.js

A simple perfomant tokenizer

Simple string tokenizer that returns both tokens and delimiters and preserves their order. Features:

Stringscanner performs lexical scanning operations on a string.

A tool that generates a TypeScript file from your Jison grammar's semantic actions so the TypeScript compiler will check those semantic actions for errors.

Yet another powerful tokenizer in js

Simple synchronous string tokenizer using Regex

Universal lexer, where you can pass your rules for lexical analytics

Simple search string tokenizer utility

Regular expressions on list of objects

Stream transform that divide the input by a sperator token

Module for fast element to selector matching (with custom :hover, :focus etc. handing)

Module for fast css selector tokenization from string