2.2.0 • Published 3 years ago
string-punctuation-tokenizer v2.2.0
string-punctuation-tokenizer
Small library that provides functions to tokenize a string into an array of words with or without punctuation
Setup
npm install string-punctuation-tokenizer
Usage
var stringTokenizer = require('string-punctuation-tokenizer');
or ES6
import {tokenize} from 'string-punctuation-tokenizer';
Tokenize with punctuation
import {tokenize} from './src/tokenizers'; // use the import from above instead of this
let words = tokenize({text: 'Hello world, my name is Manny!', includePunctuation: true});
// words = ["Hello", "world", ",", "my", "name", "is", "Manny", "!"]
Tokenize without punctuation
import {tokenize} from './src/tokenizers'; // use the import from above instead of this
let words = tokenize({text: 'Hello world, my name is Manny!'});
// words = ["Hello", "world", "my", "name", "is", "Manny"]
Documentation
See detailed documentation and live WYSIWYG playground here: https://string-punctuation-tokenizer.netlify.app/#/Tokenize
2.2.0
3 years ago
2.1.2
4 years ago
2.1.1
4 years ago
2.0.3
4 years ago
2.1.0
4 years ago
2.0.2
4 years ago
2.0.1
4 years ago
2.0.0
4 years ago
2.0.0-rc.1
4 years ago
2.0.0-beta.2
5 years ago
2.0.0-beta.1
5 years ago
2.0.0-alpha.5
5 years ago
2.0.0-alpha.4
5 years ago
2.0.0-alpha.3
5 years ago
2.0.0-alpha.2
5 years ago
2.0.0-alpha.1
5 years ago
0.9.4
5 years ago
0.9.3
5 years ago
0.8.1
6 years ago
0.9.1
6 years ago
0.9.0
6 years ago
0.8.0
6 years ago
0.7.0
6 years ago
0.6.0
6 years ago
0.5.0
6 years ago
0.4.0
6 years ago
0.3.0
6 years ago
0.2.0
6 years ago
0.1.0
6 years ago