@voxrt/kws-browser

voxrt-kws-browser

On-device 14-class keyword spotter running entirely inside a browser tab. No server, no download-first, no native install. Streaming 16 kHz mono microphone in, per-class posteriors + firing detections out.

• Status: shipping, browser-ready. Verified end-to-end via live microphone on Chrome (V8) and Safari (JSC) across desktop and mobile.
• Current version: v0.1.0
• Target: any modern browser with WebAssembly SIMD128 — Chrome 91+, Firefox 89+, Safari 16.4+, iOS Safari 16.4+, Android Chrome, Edge 91+.
• Bundle: ~190 KB WebAssembly runtime + ~1.28 MB .vxrt model = ~1.47 MB total download.
• License: Apache-2.0 (wrapper sources) · proprietary (compiled runtime, redistribution allowed via this SDK)

Vocabulary

Fires on any of these 14 English keywords (all trained on speaker-disjoint audio, synthetic + Common Voice):

yes, no, cancel, play, pause, next, previous, up, down, back, on, off, voxrt, hey_vox

The class list is not hard-coded in the SDK — it's read from the .vxrt manifest at load time via engine.classNames(), so future model rev with different vocab reuses the same runtime binary.

Positioning — read this before shipping to production

The three deployment tiers we support, most-secure first:

Tier	SDK	Model delivery	Recommended for
Native (coming — Android / iOS / Linux)	mirrors voxrt-wake-word-* shape	Embedded in the app / SDK asset, AES-256-GCM encrypted .vxrt	Production consumer apps, embedded devices, offline appliances
Browser (this SDK)	@voxrt/kws-browser (npm)	Bundled plaintext .vxrt — same model bytes as the native SDKs, without the AES-GCM envelope	Free demos, prototypes, marketing landing pages, quick internal tools
Browser + server-gated model (v0.2+ roadmap)	Same npm package	.vxrt fetched from an auth-token-gated endpoint per session — revocable + auditable	Commercial browser deployments where model IP matters more than device reach

If your product monetises a custom-vocabulary KWS (brand-specific commands, in-app control), talk to us about the native SDKs. Browsers give a low-friction demo surface and prototyping speed, not maximum-security model distribution.

What is VoxRT?

VoxRT is a from-scratch inference runtime for on-device speech models. No ONNX Runtime, no PyTorch Mobile, no LiteRT — a custom Rust core sized and tuned for streaming voice workloads. This package is the WebAssembly port of that same runtime — same .vxrt model format as our sister browser SDK @voxrt/wake-word-browser and the native voxrt-* bindings on Android, iOS, Linux, Python, Node, and Go.

Model architecture

Cache-aware streaming Conformer, 4 blocks × 4 heads (d=96, head_dim=24), 2× stride-2 subsampling stem, 636 K parameters, fp16-packed weights → 1.28 MB .vxrt.

Model quality (M13, kws-36-0.9966.ckpt):

• val/f1_macro: 0.9671 (14-class held-out set, speaker-disjoint)
• val/acc: 0.9966
• Default threshold 0.90 + 3-consecutive-frame confirmation + 25-emit (~1 s) per-class cooldown → live-mic robust to media noise and MUSAN speech/music mix.

Performance

60-second offline benchmark, WebAssembly with SIMD128, cumulative RTF (wall_time / audio_time) across the actual pushPcmI16 path:

Device	Browser	RTF	Throughput	Chunk mean
10-core desktop x86	Chrome 149	9.7 × 10⁻³	103× realtime	0.31 ms
(iPhone / Android measured on next release)

The WASM SIMD128 port covers every hot scalar in the streaming forward — MHSA matmuls, softmax, silu, glu, poly-expf, stem conv, residuals. See docs/kws/M13-BROWSER-PERF.md for the F1–F4 optimisation catalogue.

Install

npm install @voxrt/kws-browser

Or CDN (both files sit next to each other in the published package):

<script type="module">
  import init, { KwsEngine } from
    "https://unpkg.com/@voxrt/kws-browser@0.1.0/voxrt-kws-browser.js";
  await init();
  const bytes = new Uint8Array(await (await fetch(
    "https://unpkg.com/@voxrt/kws-browser@0.1.0/voxrt_kws.vxrt"
  )).arrayBuffer());
  const engine = KwsEngine.fromBytes(bytes);
  console.log("classes:", engine.classNames());
</script>

The npm package ships everything you need in one install: the WebAssembly runtime, the JavaScript glue + TypeScript types, and the KWS model file (voxrt_kws.vxrt, plaintext v2 — the browser build is compiled without a decrypt key, so there's no crypto material anywhere in the shipped binary and no risk to the native SDKs' model IP).

Quick start

The full runnable page lives at examples/live-mic-quickstart/. Minimum you need:

<!doctype html>
<html>
<body>
<button id="start">Listen</button>
<pre id="log"></pre>
<script type="module">
import init, { KwsEngine } from
  "./node_modules/@voxrt/kws-browser/voxrt-kws-browser.js";

document.getElementById("start").addEventListener("click", async () => {
  await init();
  const engine = KwsEngine.fromBytes(new Uint8Array(
    await (await fetch("voxrt_kws.vxrt")).arrayBuffer()));
  engine.threshold = 0.9;
  engine.consecutiveFramesRequired = 3;

  const stream = await navigator.mediaDevices.getUserMedia({audio: true});
  const ac = new AudioContext({sampleRate: 16000});
  const src = ac.createMediaStreamSource(stream);
  const proc = ac.createScriptProcessor(512, 1, 1);
  const mute = ac.createGain(); mute.gain.value = 0;
  const pcm = new Int16Array(512);

  proc.addEventListener("audioprocess", (e) => {
    const input = e.inputBuffer.getChannelData(0);
    for (let i = 0; i < input.length; i++) {
      const s = Math.max(-1, Math.min(1, input[i]));
      pcm[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
    }
    for (const d of engine.pushPcmI16(pcm)) {
      document.getElementById("log").textContent +=
        `"${d.class}" t=${d.timestampSec.toFixed(3)}s score=${d.score.toFixed(4)}\n`;
    }
  });
  src.connect(proc);
  proc.connect(mute);
  mute.connect(ac.destination);
});
</script>
</body>
</html>

Serve it over HTTPS (or localhost) — browsers refuse getUserMedia on plain HTTP for non-loopback origins.

API

class KwsEngine {
  static fromBytes(bytes: Uint8Array): KwsEngine;

  pushPcmI16(pcm: Int16Array): Detection[];
  pushPcmF32(pcm: Float32Array): Detection[];

  classNames(): string[];         // read from the .vxrt manifest
  numClasses(): number;
  currentPosteriors(): Float32Array;  // latest per-class sigmoid, [n_classes]
  sampleRate(): number;           // 16000

  threshold: number;                    // default 0.9
  consecutiveFramesRequired: number;    // default 3 emit frames (~120 ms)
  cooldownFrames: number;               // default 25 emits (~1.0 s post-firing)
}

interface Detection {
  class: string;                  // e.g. "yes"
  classIndex: number;             // matches engine.classNames()[i]
  emitIndex: number;              // encoder-frame counter (40 ms/frame @ 25 fps)
  timestampSec: number;
  score: number;                  // sigmoid, [0, 1]
}

function version(): string;
function archFamily(): string;    // "kws-v1"

Detection schema mirrors our sister browser package @voxrt/wake-word-browser with emitIndex in place of frameIndex (Conformer emits every 40 ms at 25 fps rather than every 10 ms at 100 fps).

Tuning

• threshold — sigmoid space [0, 1]. Default 0.9. Lower for higher recall (more triggers, more false positives); raise for higher precision. Per-class thresholds are on the v0.2 roadmap.
• consecutiveFramesRequired — number of consecutive above-threshold emits before firing. Default 3 (~120 ms at 25 fps) suppresses transient false positives on media playback.
• cooldownFrames — post-firing per-class silence in emit frames. Default 25 = 1.0 s. Prevents multi-fire on a long utterance of the same word.

Browser support

WebAssembly SIMD128 is stable in every modern browser:

Engine	Minimum version	Released
Chrome / Edge (V8)	91	May 2021
Firefox	89	June 2021
Safari (JSC) — desktop + iOS	16.4	Mar 2023

If you must support older Safari (<16.4), we don't ship a scalar-fallback build in v0.1.x — the runtime hard-requires SIMD128. Older engines will fail at init().

License

• Wrapper sources (voxrt_kws_browser crate, wasm-bindgen glue, examples): Apache-2.0.
• Compiled runtime (voxrt-kws-browser_bg.wasm and its symbol table): proprietary binary. Redistribution is allowed only as an unmodified part of this SDK package. See LICENSE-BINARY.
• KWS model weights (voxrt_kws.vxrt bundled in the npm package): proprietary in-house model, trained on synthetic (Pocket TTS) and licensed speech (Common Voice), no upstream license obligations.