Byte pair encoding Packages

Build your own vocabulary from application-specific corpus using Byte pair encoding (BPE) algorithm.

TypeScript version of PGN Tokenizer, a Byte Pair Encoding (BPE) tokenizer for Chess Portable Game Notiation (PGN).