@hyv/openai
0.9.0 • Published 10 months ago
Our library `@lenml/llama2-tokenizer` has been deprecated. We are excited to introduce our new library `@lenml/tokenizers` as its replacement, offering a broader set of features and an enhanced experience.
baichuan2 tokenizer for NodeJS/Browser
chatglm3 tokenizer for NodeJS/Browser
claude tokenizer for NodeJS/Browser
claude1 tokenizer for NodeJS/Browser
command_r_plus tokenizer for NodeJS/Browser
gemma tokenizer for NodeJS/Browser
gpt2 tokenizer for NodeJS/Browser
gpt3 tokenizer for NodeJS/Browser
gpt35turbo tokenizer for NodeJS/Browser