Tokenize | npm.io

General purpose generic matcher

matcher match tokenize search

1.3.0 • Published 4 years ago

An easy and simple way to encrypt and decrypt information for client and server side components

encryption tokenize encrypt react use client nextjs use server

1.0.4 • Published 10 months ago

estree (and esast) utility to parse from JavaScript

unist estree estree-util esast esast-util util utility js parse tokenize

2.0.1 • Published 2 years ago

Tokenize a Concatenation Expression

concat numbers string tokenize variables

0.0.1 • Published 5 years ago

get word counts / frequencies on a per-speaker or per-category basis, or as an aggregate

wordcount frequency speakers tokenize stem category nlp bayes predict tokens

0.0.1721 • Published 3 years ago

Lexer to tokenize CSS

bocss lexer tokenize css

1.0.3 • Published 4 years ago

Parse and manipulate tokenized strings with ease!

tags brackets html replace replacements replacer tokens tokenize tokenized secrets

0.0.4 • Published 2 years ago

Parse and manipulate tokenized strings with ease!

tags brackets html replace replacements replacer tokens tokenize tokenized secrets

0.0.2 • Published 2 years ago

Tokenize strings with double quotes, with embedded escaped, backslashed double quotes

tokenize double quote slashed escaped single-quote double-quote customizable additional word boundary characters single character tokens

1.0.6 • Published 4 years ago

A connector for the Trustami.AI API.

ai nlp trustami summarize sentiment language keywords named entities entities tokenize

1.0.4 • Published 2 years ago

tokenize your data securely

token cookies tokenise tokenize jwt

1.1.0 • Published 4 years ago

iterator based tokenizer for writing parsers

lexer tokenize tokenizer parse parser

3.0.1 • Published 3 years ago

Simple, but powerful lexical scanner that is a more minimal implementation of X-Scanner

y-scanner yscanner x-scanner xscanner stringscanner scanner string text textscanner lex

0.7.9 • Published 3 years ago

micromark utility to tokenize subtokens

micromark util utility tokenize

2.1.0 • Published 10 months ago

NativeExtractor bindings for Node.js

machine learning data mining datamining extraction entity text regex regexp

1.0.5 • Published 3 years ago

A Smarter String Split. Preserve Separator and more!

chop deconcat separate split string substring tokenize word

0.0.2 • Published 5 years ago

POS Tagger and lemmatizer for javascript

nlp tokenizer tokenize pos tagger Part-Of-Speech Tagger lemma lemmatizer javascript

1.0.2 • Published 1 year ago

String ngram splitter.

ngram split search tokenize tokenizer convert converter parse parser

0.1.3 • Published 9 months ago

Transform hypertext strings (e.g., HTML, Markdown) into plain text for natural language processing (NLP) normalization

html markdown plaintext text token tokenize normalize strip nlp

1.0.1 • Published 1 year ago

Tokenizes a text using regex rules and returns the tokens with their positions in the text given.

tokenizer tokenize parser regex regexp token tokens

1.0.6 • Published 3 years ago

16 7 8

Tokenize Packages

generic-matcher

easy-simple-encryption

esast-util-from-js

deconcat

bayes-tokens

bocss-lexer

bracketter

bracketter-js

dqtokenizer

trustami-ai

tokie

tokenizer-next

y-scanner

micromark-util-subtokenize

nativeextractor.js

separo

nlp-js-tools-english

ngramjs

nomark

positional-tokenizer