# @csstools/css-tokenizer

> Tokenize CSS

Latest version **4.0.1** (published 2026-09-18) · MIT license · 0 weekly downloads

## Install

```sh
npm install @csstools/css-tokenizer
pnpm add @csstools/css-tokenizer
yarn add @csstools/css-tokenizer
bun add @csstools/css-tokenizer
```

## Health

**Score 60/100 (C)** — status: active.

Positive: esm support; no vulnerabilities; recently updated; high maintenance score.

Warnings: low downloads; no types.

## Facts

| | |
|---|---|
| Version | 4.0.1 |
| Published | 2026-09-18 |
| First published | 2022-11-14 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | ESM |
| Node | >=20.19.0 |
| Dependencies | 0 |
| Unpacked size | 40 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 1053 |
| Maintainers | jonathantneal, alaguna, romainmenke |
| Keywords | css, tokenizer |

## Links

- npm: https://www.npmjs.com/package/@csstools/css-tokenizer
- Repository: https://github.com/csstools/postcss-plugins
- Homepage: https://github.com/csstools/postcss-plugins/tree/main/packages/css-tokenizer#readme
- Issues: https://github.com/csstools/postcss-plugins/issues
- Funding: https://github.com/sponsors/csstools
- npm.io page: https://npm.io/package/@csstools/css-tokenizer

## Alternatives

- [style-dictionary](https://npm.io/package/style-dictionary.md) — 2.0M weekly downloads
- [postcss-merge-idents](https://npm.io/package/postcss-merge-idents.md) — 1.7M weekly downloads
- [@fontsource/noto-sans](https://npm.io/package/@fontsource/noto-sans.md) — 93.0K weekly downloads
- [uglifycss](https://npm.io/package/uglifycss.md) — 71.6K weekly downloads
- [mat4-interpolate](https://npm.io/package/mat4-interpolate.md) — 23.3K weekly downloads

## Recent versions

- 4.0.1 (latest) — 2026-09-18
- 4.0.0 — 2026-01-14
- 3.0.4 — 2025-05-27
- 3.0.3 — 2024-10-25
- 3.0.2 — 2024-10-10
- 3.0.1 — 2024-08-18
- 3.0.0 — 2024-08-03
- 2.4.1 — 2024-07-05
- 2.4.0 — 2024-07-05
- 2.3.3 — 2024-07-03
- 2.3.2 — 2024-06-29
- 2.3.1 — 2024-05-04
- 2.3.0 — 2024-05-04
- 2.2.4 — 2024-03-13
- 2.2.3 — 2023-12-31
- … 9 more at https://npm.io/package/@csstools/css-tokenizer/versions

## README

# CSS Tokenizer <img src="https://cssdb.org/images/css.svg" alt="for CSS" width="90" height="90" align="right">

[<img alt="npm version" src="https://img.shields.io/npm/v/@csstools/css-tokenizer.svg" height="20">][npm-url]
[<img alt="Build Status" src="https://github.com/csstools/postcss-plugins/actions/workflows/test.yml/badge.svg?branch=main" height="20">][cli-url]
[<img alt="Discord" src="https://shields.io/badge/Discord-5865F2?logo=discord&logoColor=white">][discord]

Implemented from : https://drafts.csswg.org/css-syntax/

## API

[Read the API docs](./docs/css-tokenizer.md)

## Usage

Add [CSS Tokenizer] to your project:

```bash
npm install @csstools/css-tokenizer --save-dev
```

```js
import { tokenize } from '@csstools/css-tokenizer';

const myCSS =  `@media only screen and (min-width: 768rem) {
	.foo {
		content: 'Some content!' !important;
	}
}
`;

const tokens = tokenize({
	css: myCSS,
});

console.log(tokens);
```

Or use the streaming interface:

```js
import { tokenizer, TokenType } from '@csstools/css-tokenizer';

const myCSS = `@media only screen and (min-width: 768rem) {
	.foo {
		content: 'Some content!' !important;
	}
}
`;

const t = tokenizer({
	css: myCSS,
});

while (true) {
	const token = t.nextToken();
	if (token[0] === TokenType.EOF) {
		break;
	}

	console.log(token);
}
```

### Options

```ts
{
	onParseError?: (error: ParseError) => void
}
```

#### `onParseError`

The tokenizer is forgiving and won't stop when a parse error is encountered.

To receive parsing error information you can set a callback.

```js
import { tokenizer, TokenType } from '@csstools/css-tokenizer';

const t = tokenizer({
	css: '\\',
}, { onParseError: (err) => console.warn(err) });

while (true) {
	const token = t.nextToken();
	if (token[0] === TokenType.EOF) {
		break;
	}
}
```

Parser errors will try to inform you where in the tokenizer logic the error happened.
This tells you what kind of error occurred.

## Order of priorities

1. specification compliance
2. correctness
3. reliability
4. tokenizing and serializing must round trip losslessly
5. exposing useful aspects about the source code
6. runtime performance
7. package size


[cli-url]: https://github.com/csstools/postcss-plugins/actions/workflows/test.yml?query=workflow/test
[discord]: https://discord.gg/bUadyRwkJS
[npm-url]: https://www.npmjs.com/package/@csstools/css-tokenizer

[CSS Tokenizer]: https://github.com/csstools/postcss-plugins/tree/main/packages/css-tokenizer

---
_Source: https://npm.io/package/@csstools/css-tokenizer · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._