intent-tools v0.1.0
Intent tools
Tools for processing intents from RASA JSONs to Facebook fast-text learning sets.
Because there is good GUI: Rasa NLU Trainer
CLI Usage
- Convert RASA json to fast-text learning set
$ intools jsonToText ./testData.json ./testData.txt
- Convert RASA json to fast-text learning set and multiply by entities
$ intools jsonToText -m ./testData.json ./testData.txt
- Make word vectors learning set from wiki XML export
$ intools wikiToText ./testData.xml ./testData.txt
API
Classes
Functions
MultiplicatorStream
Kind: global class
new MultiplicatorStream()
Multiplicates a learning set data with available entities information
Example
const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');
const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');
const entities = new main.EntitiesFromJson(from);
const pipeline = [
new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];
entities.loadEntities()
.then(() => main.jsonToText(from, to, pipeline))
.catch(e => console.error(e));
new MultiplicatorStream(getVariants)
Param | Type |
---|---|
getVariants | function |
MultiplicatorStream
Kind: global class
new MultiplicatorStream()
Multiplicates a learning set data with available entities information
Example
const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');
const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');
const entities = new main.EntitiesFromJson(from);
const pipeline = [
new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];
entities.loadEntities()
.then(() => main.jsonToText(from, to, pipeline))
.catch(e => console.error(e));
new MultiplicatorStream(getVariants)
Param | Type |
---|---|
getVariants | function |
Pipeline
Kind: global class
- Pipeline
- new Pipeline()
- .add(pipe) ⇒ this
- .promise() ⇒ promise
new Pipeline()
Simple tool, which creates a Promise from pipeline of streams
pipeline.add(pipe) ⇒ this
Append a stream
Kind: instance method of Pipeline
Param | Type | Description |
---|---|---|
pipe | ReadableStream | Writable | the transform stream |
pipeline.promise() ⇒ promise
Get a promise
Kind: instance method of Pipeline
jsonToText(input, output, pipeline, mapFn) ⇒ Promise
Create fast-text learning set from Rasa intents json
Kind: global function
Param | Type | Default | Description |
---|---|---|---|
input | string | ReadableStream | path of Rasa intent set or stream | |
output | string | Writable | path or stream to write fast-text learning set | |
pipeline | Array | array of transform streams to modify the learning set | |
mapFn | function | text normalizer function |
Example
const path = require('path');
const { jsonToText } = require('intent-tools');
const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');
main.jsonToText(from, to)
.catch(e => console.error(e));
wikiToText(input, output, mapFn) ⇒ Promise
Create a pretrained word vectors learning set from Wikipedia XML dump
Kind: global function
Param | Type | Default | Description |
---|---|---|---|
input | string | ReadableStream | path of Rasa intent set or stream | |
output | string | Writable | path or stream to write fast-text learning set | |
mapFn | function | text normalizer function |
normalize(str)
Preserves only letters (with or withour diacritics) and makes everything lowercased
Kind: global function Returs: string
Param | Type | Description |
---|---|---|
str | string | input string |
7 years ago