Tokenize | npm.io

Simple, but powerful lexical scanner that is a more minimal implementation of X-Scanner

y-scanner yscanner x-scanner xscanner stringscanner scanner string text textscanner lex

0.7.9 • Published 3 years ago

docast utility to parse docblocks

comment docast docblock documentation jsdoc markdown mdast parse tsdoc typedoc

1.0.0-alpha.1 • Published 1 year ago

An abstract tokenizer.

tokenizer tokenize parse parser

2.1.0 • Published 10 months ago

Easily assert that values are NonNullable or Nullable.

tokenizer tokenize

1.0.0 • Published 11 months ago

A gemtext (`text/gemini`) parser with support for streaming, ASTs, and CSTs

dioscuri castor pollux gemini gemtext document parse tokenize lexer stream

1.1.0-laosbpatch.0 • Published 4 years ago

A NodeJS library to tokenize strings

javascript js nodejs node token tokenize tokenizer

0.1.4 • Published 5 years ago

A General Purpose Toolkit Library for Javascript

javascript js nodejs node json ini clone merge parse match

0.0.22 • Published 3 years ago

Uses snapdragon to tokenize a single JavaScript block comment into an object, with description, tags, and code example sections that can be passed to any other comment parsers for further parsing.

code comment examples gfm indented javadoc javascript jsdoc parse tokenize

3.0.1 • Published 2 years ago

Abstract TypeScript template engine

typescript javascript abstract modular string template engine load transpile token

1.0.0 • Published 3 years ago

JavaScript library for tokenizing strings with regular expressions

lexer tokenizer syntax lex tokenize token parsing

2.3.0-alpha.8 • Published 2 years ago

Tokenize Excel formulas

excel formula parse tokenize

2.3.6 • Published 3 years ago

Have you ever been in need of a tag box that can provide many built in functionalities that suites you needs? TagBox is built with Angular 4+ code and is developed specifically without needing to use ngModel to avoid an existing bug in Angular which makes

angular 6+tags tagger tagging tokenize tokenizer widget autocomplete tagger image tagger audio tagger

3.0.0 • Published 1 year ago

micromark utility to tokenize subtokens

micromark util utility tokenize

2.1.0 • Published 5 months ago

NativeExtractor bindings for Node.js

machine learning data mining datamining extraction entity text regex regexp

1.0.5 • Published 3 years ago

A Smarter String Split. Preserve Separator and more!

chop deconcat separate split string substring tokenize word

0.0.2 • Published 4 years ago

POS Tagger and lemmatizer for javascript

nlp tokenizer tokenize pos tagger Part-Of-Speech Tagger lemma lemmatizer javascript

1.0.2 • Published 12 months ago

String ngram splitter.

ngram split search tokenize tokenizer convert converter parse parser

0.1.3 • Published 4 months ago

Transform hypertext strings (e.g., HTML, Markdown) into plain text for natural language processing (NLP) normalization

html markdown plaintext text token tokenize normalize strip nlp

1.0.1 • Published 11 months ago

Tokenizes a text using regex rules and returns the tokens with their positions in the text given.

tokenizer tokenize parser regex regexp token tokens

1.0.6 • Published 2 years ago

A moo compatible tokenizer/lexer generator, sacrificing some performance for features.

lexer tokenize parsing

0.1.5 • Published 2 years ago