# charabia-js

> A WebAssembly binding for the charabia multilingual text tokenizer used by Meilisearch.

Latest version **0.2.0** (published 2024-11-22) · MIT license · 0 weekly downloads

## Install

```sh
npm install charabia-js
pnpm add charabia-js
yarn add charabia-js
bun add charabia-js
```

## Health

**Score 40/100 (D)** — status: stable.

Positive: has types; no vulnerabilities.

Warnings: low downloads; no esm support; large bundle; pre 1.0.

Negative: stale.

## Facts

| | |
|---|---|
| Version | 0.2.0 |
| Published | 2024-11-22 |
| First published | 2024-11-22 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | bundled |
| Module format | CommonJS |
| Dependencies | 0 |
| Unpacked size | 73.2 MB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 1 |
| Maintainers | codytseng |
| Keywords | multilingual, tokenizer, tokenization, segment, segmentation, wasm, charabia, meilisearch |

## Links

- npm: https://www.npmjs.com/package/charabia-js
- Repository: https://github.com/codytseng/charabia-js
- Homepage: https://github.com/codytseng/charabia-js#readme
- Issues: https://github.com/codytseng/charabia-js/issues
- npm.io page: https://npm.io/package/charabia-js

## Alternatives

- [jsforce](https://npm.io/package/jsforce.md) — 851.2K weekly downloads
- [react-native-qrcode-svg](https://npm.io/package/react-native-qrcode-svg.md) — 693.5K weekly downloads
- [@salesforce/plugin-data](https://npm.io/package/@salesforce/plugin-data.md) — 394.9K weekly downloads
- [@backstage/plugin-search-common](https://npm.io/package/@backstage/plugin-search-common.md) — 308.5K weekly downloads
- [@chain-registry/types](https://npm.io/package/@chain-registry/types.md) — 38.4K weekly downloads

## Recent versions

- 0.2.0 (latest) — 2024-11-22
- 0.1.0 — 2024-11-22

## README

# charabia-js

`charabia-js` is a WebAssembly binding for the [charabia](https://github.com/meilisearch/charabia) multilingual text tokenizer used by [Meilisearch](https://github.com/meilisearch/meilisearch).

## Supported scripts / languages

- Latin
- Latin - German
- Greek
- Cyrillic - Georgian
- Chinese CMN 🇨🇳
- Hebrew 🇮🇱
- Arabic
- Japanese 🇯🇵
- Korean 🇰🇷
- Thai 🇹🇭
- Khmer 🇰🇭

More information about the supported scripts and languages can be found in the [here](https://github.com/meilisearch/charabia#supported-languages).

## Installation

```sh
npm install charabia-js
```

## Usage

### Segmentation

```ts
import { segment } from "charabia-js";

console.log(segment("Hello, world!")); // [ 'Hello', ', ', 'world', '!' ]
console.log(segment("你好，世界！")); // [ '你好', '，', '世界', '！' ]
console.log(segment("Hello, 世界!")); // [ 'Hello', ', ', '世界', '!' ]
```

### Tokenization

```ts
import { tokenize, TokenKind } from "charabia-js";
import assert from "node:assert";

const tokens = tokenize(
  "The quick (\"brown\") fox can't jump 32.3 feet, right? Brr, it's 29.3°F"
);

let token = tokens[0];
assert.equal(token.lemma, "the");
assert.equal(token.kind, TokenKind.Word);

token = tokens[1];
assert.equal(token.lemma, " ");
assert.equal(token.kind, TokenKind.SoftSeparator);

token = tokens[2];
assert.equal(token.lemma, "quick");
assert.equal(token.kind, TokenKind.Word);
```

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

---
_Source: https://npm.io/package/charabia-js · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._