@jill64/cf-tiktoken NPM

@jill64/cf-tiktoken

⏳ js-tiktoken on Cloudflare Pages

Currently js-tiktoken does not work with Cloudflare Pages due to worker size limitations.
This library uses conditional export to exclude js-tiktoken implementations from Cloudflare worker bundles.

!IMPORTANT This is not the library that makes js-tiktoken work on Cloudflare Workers.

Installation

npm i @jill64/cf-tiktoken

// This function can be imported by both edge and browser.
// however, on edge it returns invalid output regardless of the input value.
import { encodingForModel } from '@jill64/cf-tiktoken'

export const fn = () => {
  const enc = encodingForModel('gpt-4')
  // ...
}