# tokenize-monster

> Yet another powerful tokenizer in js

Latest version **0.3.4** (published 2017-06-30) · MIT license · 0 weekly downloads

## Install

```sh
npm install tokenize-monster
pnpm add tokenize-monster
yarn add tokenize-monster
bun add tokenize-monster
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support; pre 1.0.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 0.3.4 |
| Published | 2017-06-30 |
| First published | 2017-05-09 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 0 |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 0 |
| Author | clitetailor |
| Maintainers | clitetailor |
| Keywords | tokenizer, parser, replacer |

## Links

- npm: https://www.npmjs.com/package/tokenize-monster
- Repository: https://github.com/clitetailor/tokenize-monster
- Homepage: https://github.com/clitetailor/tokenize-monster#readme
- Issues: https://github.com/clitetailor/tokenize-monster/issues
- npm.io page: https://npm.io/package/tokenize-monster

## Alternatives

- [babylon](https://npm.io/package/babylon.md) — 5.1M weekly downloads
- [csscolorparser](https://npm.io/package/csscolorparser.md) — 3.7M weekly downloads
- [expr-eval-fork](https://npm.io/package/expr-eval-fork.md) — 1.5M weekly downloads
- [@leeoniya/ufuzzy](https://npm.io/package/@leeoniya/ufuzzy.md) — 247.7K weekly downloads
- [xml-parser](https://npm.io/package/xml-parser.md) — 78.4K weekly downloads

## Recent versions

- 0.3.4 (latest) — 2017-06-30
- 0.3.3 — 2017-06-30
- 0.3.2 — 2017-06-30
- 0.3.1 — 2017-06-26
- 0.3.0 — 2017-06-26
- 0.2.0 — 2017-06-23
- 0.1.0 — 2017-06-22
- 0.0.1 — 2017-05-09

## README

Tokenize Monster
================

> Yet another powerful tokenizer in **JS**.

Installation
------------

```bash
npm install --save tokenize-monster
```

Getting Started
---------------

```javascript
const tokenizer = require('tokenize-monster');

const tokens = tokenizer.matchAndMap(
	'i\'d like to <drink> some <milk>',
	/<[^><]+>/,
	text => [[text]],
	text => [text]
);

console.log(tokens);
// => [ 'i\'d like to ', [ '<drink>' ], ' some ', [ '<milk>' ], '' ]

const nTokens = tokenizer.matchAndMap(
	'i\'d like to <drink> some <milk>',
	/<[^><]+>/,
	text => [text],
	text => []
);

console.log(nTokens);
// => [ '<drink>', '<milk>' ]
```

Example
-------

- [Simple XML Parser](https://github.com/clitetailor/simple-xml-parser)

Documentation
-------------

Our docs are available at [Tokenize Monster Wiki](https://github.com/clitetailor/tokenize-monster/wiki).

Changelog
---------

Visit [Github Releases](https://github.com/clitetailor/tokenize-monster/releases) page for more information.

---
_Source: https://npm.io/package/tokenize-monster · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._