speaklet

speaklet

An on-device voice-challenge toolkit for browsers and WebViews. speaklet joins offline speech recognition, microphone capture, enrollment, speaker verification, and speaker identification behind one application API.

speaklet is built on Vosklet. Speech-to-text is provided by the bundled monosklet engine, which vendors the Vosklet Vosk + Kaldi WebAssembly runtime from the Devrax/Vosklet fork. speaklet adds capture and NeXt-TDNN speaker verification through @jaehyun-ko/speaker-verification and onnxruntime-web.

Everything runs on the device. You decide whether the speech and speaker models are bundled with the application or downloaded and cached on first use.

Install

npm install speaklet
# or
pnpm add speaklet

Installing monosklet separately is not required. Its worker engine and single-thread Vosklet runtime are bundled into speaklet.

Required application setup

speaklet needs three runtime assets:

1. A Vosk speech model repackaged as USTAR TAR.
2. A NeXt-TDNN ONNX speaker model, either from the default remote URL or your own URL.
3. The onnxruntime-web Wasm binaries served by your application.

1. Prepare a speech model

Choose a language from alphacephei.com/vosk/models, then convert the downloaded ZIP archive:

unzip vosk-model-small-es-0.42.zip
cd vosk-model-small-es-0.42
tar --format=ustar -cf ../es-small.tar .

Put the TAR file in your application's static assets, for example public/models/es-small.tar. Android application assets should use plain .tar, not .tar.gz.

2. Serve the ONNX Runtime Wasm files

Copy these files from node_modules/onnxruntime-web/dist/ into a static directory served as /ort/:

ort-wasm.wasm
ort-wasm-simd.wasm

For example, a Vite application can copy them into public/ort/ as part of its setup or build script. If they are served elsewhere, pass an absolute verifier.wasmPaths URL.

3. Configure the ort bundler alias

@jaehyun-ko/speaker-verification imports an external dependency named ort. Map it to onnxruntime-web in the consuming application:

// vite.config.js
import { defineConfig } from "vite";

export default defineConfig({
  resolve: {
    alias: {
      ort: "onnxruntime-web"
    }
  }
});

Other bundlers need the equivalent alias. The repository's Examples/demo/vite.config.js shows a complete Vite plugin that serves the files in development and copies them into production builds.

Quick start: enroll, then verify

This example records two separate utterances: the first enrolls a voice, and the second verifies it.

import { createVoskletSpeaker } from "speaklet";

const enrollmentText =
  "Confirmo que esta es mi voz y autorizo guardarla como referencia";

const speaker = await createVoskletSpeaker({
  model: {
    url: "/models/es-small.tar",
    id: "vosk-model-small-es-0.42"
  },
  verifier: {
    model: "standard-384",
    wasmPaths: new URL("/ort/", window.location.origin).href
  },
  capture: {
    speechThreshold: 0.015,
    stopAfterSpoken: 3_000
  }
});

async function recordOnce() {
  const capture = await speaker.record();
  const recording = await capture.result;

  if (!recording) {
    throw new Error("Recording was cancelled");
  }

  return recording;
}

try {
  // Optional: download or restore the ONNX model before the user records.
  await speaker.warmUp((source) => console.log(`Speaker model: ${source}`));

  // 1. Enrollment recording.
  const enrollmentRecording = await recordOnce();
  const { text: enrollmentTranscript } =
    await speaker.transcribe(enrollmentRecording);

  const enrollment = await speaker.enroll(
    {
      wav: enrollmentRecording.wav,
      text: enrollmentTranscript
    },
    {
      expectedText: enrollmentText,
      matchThreshold: 0.65,
      id: "current-user",
      label: "Current user"
    }
  );

  if (!enrollment.accepted) {
    throw new Error(`Enrollment reading matched only ${enrollment.overlap}`);
  }

  // 2. A different, later recording.
  const verificationRecording = await recordOnce();
  const { text } = await speaker.transcribe(verificationRecording);
  const verification = await speaker.verify(verificationRecording.wav, {
    id: "current-user"
  });

  console.log({ text, ...verification });
  // { text, id, label, score, match, threshold }
} finally {
  await speaker.dispose();
}

record() auto-stops after speech followed by silence. Its returned handle also provides stop() for a stop button and cancel() to discard the recording. The WAV is created before transcribe() transfers the PCM block buffers to the worker, so it remains available for enroll(), verify(), or identify().

For a complete interface with UI state, progress, errors, manual stop, and model status, see Examples/demo/src/speaker.js.

Which speaker engine is being used?

speaklet contains two distinct speaker-processing paths:

Path	Speech-to-text	Speaker embeddings	What you implement
speaker.enroll(), verify(), identify()	Vosk + Kaldi through monosklet	External NeXt-TDNN ONNX models	High-level API handles enrollment, persistence, comparison, and thresholds
createVoskletMonoWorker() / engine.loadSpkModel()	Vosk + Kaldi through monosklet	Native Vosk x-vectors	Your application stores, averages, and compares raw embeddings

Therefore, speaklet's high-level speaker methods do not use monosklet's native Vosk speaker-recognition model. Vosk + Kaldi recognizes the words; @jaehyun-ko/speaker-verification and onnxruntime-web produce and compare the voice embeddings.

The bundled monosklet functionality is not restricted. speaklet re-exports createVoskletMonoWorker() and supportsWorkerHost() for applications that want the native Vosk x-vector API. The default suite's worker is also available at speaker.engine at run time, but no high-level speaklet method wraps loadSpkModel().

Configure the models

Vosk speech model

Use a local model for a fully offline installation:

const speaker = await createVoskletSpeaker({
  model: {
    url: "/models/es-small.tar",
    id: "vosk-model-small-es-0.42",
    storagePath: "Spanish"
  }
});

Or download one from your own server on first use:

const speaker = await createVoskletSpeaker({
  model: {
    url: "https://cdn.example.com/vosk-model-small-es-0.42.tar.gz",
    id: "vosk-model-small-es-0.42"
  }
});

The archive is cached under id; change the ID when replacing model contents at the same URL.

NeXt-TDNN speaker model

The following aliases are provided by @jaehyun-ko/speaker-verification:

Alias	Approximate download	Notes
standard-384	27 MB	Default; highest-capacity standard option
standard-256	28 MB	Standard model
standard-192	16 MB	Smaller standard model
standard-128	7.5 MB	Compact standard model
mobile-256	20 MB	Mobile architecture
mobile-128	5 MB	Smallest; useful for WebView applications

With only an alias, the model is downloaded from the project's default Hugging Face repository and cached on first use:

verifier: { model: "mobile-128", wasmPaths: "/ort/" }

To avoid a runtime dependency on Hugging Face, host the matching ONNX file:

verifier: {
  model: "standard-384",
  modelUrl: "/models/NeXt_TDNN_C384_B1_K65_7.onnx",
  wasmPaths: new URL("/ort/", window.location.origin).href
}

Keep the alias consistent with the selected model file. Embeddings from different models are not comparable, and changing models requires enrollment again. wasmPaths should be absolute when the application has pages below a subpath such as /speaker/.

Multiple speakers

Enroll each person under a stable ID, optionally with a display label:

await speaker.enroll(
  { wav: rafaRecording.wav, text: rafaTranscript },
  { id: "rafa", label: "Rafael", expectedText: enrollmentText }
);

await speaker.enroll(
  { wav: anaRecording.wav, text: anaTranscript },
  { id: "ana", label: "Ana", expectedText: enrollmentText }
);

const identified = await speaker.identify(probeRecording.wav);
// {
//   id: "ana", label: "Ana", score: 0.71, match: true, threshold: 0.5,
//   scores: [{ id: "ana", ... }, { id: "rafa", score: 0.22, ... }]
// }

speaker.listSpeakers();
await speaker.verify(probeRecording.wav, { id: "rafa" }); // one-to-one
speaker.clearReference("ana");
speaker.clearAllReferences();

Calls without an ID use the "default" speaker.

Persistence and privacy

Reference embeddings are stored in localStorage by default. Enrollment audio is not persisted unless explicitly requested:

await speaker.enroll(
  { wav: recording.wav, text },
  {
    id: "rafa",
    label: "Rafael",
    expectedText: enrollmentText,
    persist: true
  }
);

const wav = await speaker.loadEnrollmentAudio("rafa");
await speaker.clearEnrollmentAudio("rafa"); // keeps the embedding
speaker.clearReference("rafa");             // removes both

Audio persistence uses Cache Storage and is best-effort. On origins where that API cannot be used, such as some capacitor:// environments, enrollment still succeeds and loadEnrollmentAudio() returns null.

Voice recordings and embeddings are biometric data. Tell users what is stored, protect application access, and provide a way to clear it.

Quick API guide

High-level suite

const speaker = await createVoskletSpeaker({
  model: { url, id, storagePath },
  engine,         // optional existing monosklet-compatible engine
  engineOptions,  // options used when speaklet creates its worker
  verifier,       // NeXt-TDNN/ONNX options
  capture         // defaults merged into every record() call
});

API	Purpose
speaker.warmUp(onStatus?)	Load the ONNX speaker model before first use
speaker.record(options?)	Start microphone capture; returns a capture handle
speaker.transcribe(recording, options?)	Recognize the captured PCM through monosklet
speaker.enroll(input, options?)	Gate an enrollment by expected text and save its embedding
speaker.verify(audio, options?)	Compare audio with one enrolled speaker
speaker.identify(audio, options?)	Rank audio against every enrolled speaker
speaker.listSpeakers()	Return { id, label? } entries
speaker.hasReference(id?)	Check whether a speaker is enrolled
speaker.loadEnrollmentAudio(id?)	Load audio saved with persist: true
speaker.clearEnrollmentAudio(id?)	Remove saved audio but keep its embedding
speaker.clearReference(id?)	Remove one embedding and its saved audio
speaker.clearAllReferences()	Remove every enrollment
speaker.dispose()	Close the shared AudioContext and owned worker engine

enroll() resolves { accepted, overlap, embedding? }. verify() resolves { id, label?, score, match, threshold }. identify() adds a best-first scores array.

Capture handle

API	Purpose
capture.result	Resolves to { blocks, wav, sampleRate, reason }, or null after cancellation
capture.stop()	Finish immediately and return the audio captured so far
capture.cancel()	Discard the recording and resolve result with null

record() accepts an existing stream or audioContext. Values provided by the application are left open during capture teardown.

Lower-level exports

Export	Purpose
startCapture(engine, options)	Use speaklet capture with your own compatible engine
createSpeakerVerifier(options)	Use NeXt-TDNN enrollment and comparison without speech recognition
createLocalStorageReferenceStore(key)	Create the default-style multi-speaker store
encodeWav(blocks, sampleRate)	Encode mono Float32 PCM as 16-bit WAV
normalizeText, textsMatch, wordOverlap	Compare challenge transcripts despite case, accents, and punctuation
createVoskletMonoWorker, supportsWorkerHost	Access the bundled monosklet worker directly
DEFAULT_SAME_SPEAKER_THRESHOLD, DEFAULT_SPEAKER_ID	Default verification constants

createSpeakerVerifier() additionally exposes init, embed, compare, enroll, verify, identify, reference management, and persisted-enrollment audio management. The published TypeScript declarations document the complete option and result shapes.

Run the repository demo

From a fresh clone of the monorepo:

pnpm run setup
pnpm run demo

setup installs dependencies, builds the packages, packs the tarballs consumed by the demo, and downloads the uncommitted model assets. Open the speaker/ page for the high-level speaklet flow or spk/ for native Vosk x-vectors.

License and third-party software

MIT—see LICENSE.

The Vosklet lineage and the Vosk, Kaldi, OpenFST, OpenBLAS, NeXt-TDNN, and ONNX Runtime components are documented in THIRD_PARTY_NOTICES.md, which ships in the npm package. Models have their own licenses; verify them before redistribution.