nai-js-tokenizer
BPE tokenizer used for the NovelAI frontend.
1.1.0 • Published 2 years ago
Sentencepiece Packages
sentencepiece-js
Sentencepiece tokenization for natural language processing, JS version.
@lambda_x/sptk
Sentencepiece tokenization for natural language processing, JS version.
@agnai/sentencepiece-js
Sentencepiece tokenization for natural language processing, JS version.
@sctg/sentencepiece-js
Sentencepiece tokenization for natural language processing, JS version.