# js-tiktoken

> JavaScript port of tiktoken

Latest version **1.0.21** (published 2025-08-09) · MIT license · 0 weekly downloads

## Install

```sh
npm install js-tiktoken
pnpm add js-tiktoken
yarn add js-tiktoken
bun add js-tiktoken
```

## Health

**Score 40/100 (D)** — status: maintenance-mode.

Positive: has types; esm support; no vulnerabilities; high quality score.

Warnings: low downloads; large bundle.

Negative: stale; low maintenance score.

## Facts

| | |
|---|---|
| Version | 1.0.21 |
| Published | 2025-08-09 |
| First published | 2023-05-13 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | bundled |
| Module format | ESM + CommonJS |
| Dependencies | 1 |
| Unpacked size | 21.4 MB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 1074 |
| Maintainers | davidduong |

## Links

- npm: https://www.npmjs.com/package/js-tiktoken
- Repository: https://github.com/dqbd/tiktoken
- Homepage: https://github.com/dqbd/tiktoken#readme
- Issues: https://github.com/dqbd/tiktoken/issues
- npm.io page: https://npm.io/package/js-tiktoken

## Dependencies (1)

- [base64-js](https://npm.io/package/base64-js.md) ^1.5.1

## Recent versions

- 1.0.21 (latest) — 2025-08-09
- 1.0.7 (devel) — 2023-06-15
- 1.0.20 — 2025-04-22
- 1.0.19 — 2025-02-14
- 1.0.18 — 2025-02-04
- 1.0.17 — 2025-02-02
- 1.0.16 — 2024-12-19
- 1.0.15 — 2024-10-04
- 1.0.14 — 2024-08-15
- 1.0.13 — 2024-08-15
- 1.0.12 — 2024-05-13
- 1.0.11 — 2024-04-12
- 1.0.10 — 2024-01-28
- 1.0.9 — 2024-01-28
- 1.0.8 — 2023-11-15
- … 7 more at https://npm.io/package/js-tiktoken/versions

## README

# ⏳ js-tiktoken

tiktoken is a [BPE](https://en.wikipedia.org/wiki/Byte_pair_encoding) tokeniser for use with
OpenAI's models. This is a pure JS port of the original tiktoken library.

Install the library from NPM:

```
npm install js-tiktoken
```

## Lite

You can only load the ranks you need, which will significantly reduce the bundle size:

```typescript
import { Tiktoken } from "js-tiktoken/lite";
import o200k_base from "js-tiktoken/ranks/o200k_base";

const enc = new Tiktoken(o200k_base);
assert(enc.decode(enc.encode("hello world")) === "hello world");
```

Alternatively, encodings can be loaded dynamically from our CDN hosted on Cloudflare Pages.

```typescript
import { Tiktoken } from "js-tiktoken/lite";

const res = await fetch(`https://tiktoken.pages.dev/js/o200k_base.json`);
const o200k_base = await res.json();

const enc = new Tiktoken(o200k_base);
assert(enc.decode(enc.encode("hello world")) === "hello world");
```

## Full usage

If you need all the OpenAI tokenizers, you can import the entire library:

> [!CAUTION]
> This will include all the OpenAI tokenizers, which may significantly increase the bundle size. See

```typescript
import assert from "node:assert";
import { getEncoding, encodingForModel } from "js-tiktoken";

const enc = getEncoding("gpt2");
assert(enc.decode(enc.encode("hello world")) === "hello world");
```

---
_Source: https://npm.io/package/js-tiktoken · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._