0.1.1 • Published 1 year ago
@completion/sampling v0.1.1
sampling
Common library for model sampling.
Installation:
Run
yarn add @completion/sampling
Usage:
For the most basic usage, you can start from the following snippet:
import { createSimpleSamplingStream, defaultModelParams } from '@completion/sampling';
createSimpleSamplingStream({
basicAuth: `Basic ${someAccessToken}`,
prompt: "\n\nHuman: What is the meaning of life? \n\nAssistant:",
model: {
// The `defaultModelParams` object is provided for convenience, with
// a set of sane defaults for model params:
...defaultModelParams,
tokenLimitPerResponse: 256,
stopSequence: ["\n\nHuman:", "\n\nAssistant:"],
name: 'model-name',
},
onSampleComplete: (completion) => {
console.log('Complete sample:', completion);
},
onSampleChange: (completion) => {
console.log('Sample so far:', completion);
},
onError: (err) => {
console.log('Error:', err);
},
});
If you're using the interface with conversation and templates instead of raw prompts (not recommended for most users), use:
import { createSamplingStream, defaultModelParams } from '@completion/sampling';
// createSamplingStream returns an object containing the WebSocket instance:
const { client } = createSamplingStream({
basicAuth: `Basic ${someAccessToken}`,
convo: conversation,
template: 'default',
model: {
// The `defaultModelParams` object is provided for convenience, with
// a set of sane defaults for model params:
...defaultModelParams,
name: 'model-name',
},
onSampleComplete: (data) => {
console.log('Model says:', data.sampleResult);
},
});
Additional options are available for the method:
createSamplingStream({
basicAuth,
convo,
template,
model,
// A full url to be used for the WebSocket connection:
url: 'https://foo.bar/foo/bar',
// A WebSocket instance. This causes url above to be ignored completely:
ws: new WebSocket('http://foo/bar),
onSampleComplete,
// Called every time new sampling data arrives, with the same signature as onSampleComplete:
//
// Returning `false` from this method will stop sampling, and cause `onSampleComplete` to be
// called with `stopReason` set to `stopCallback`.
onSampleChange: (data) => {},
// Called with an Error object, if one arises during sampling:
onError: (error) => {},
// Called whenever the client internally logs something. If unset, `console.log/info/warn` is
// used instead.
onLog: (type: 'info' | 'warn', message) => {},
});
A promise-based interface is also available, which resolves once sampling is complete:
const sampleResult = await createSamplingStreamPromise({
/* same options as above */,
rejectOnError: true
});
If provided, callback methods will still be called, before the promise returns/rejects (if appropriate).