@jcrawley_dev/firevad v0.9.6

Firevad - Voice Activity Detector for Firefox Browser Add-Ons

Forked from the ricky0123/vad project, this modified 'web' package prompts the user for microphone permissions and run callbacks on segments of audio with user speech. This package has been specifically adapted to work in Firefox Add-Ons. If you need this functionality on Chrome Extensions or elsewhere, it is recommended to use the original project.

Firevad has a modified 'web' package, and differs from the original VAD in the following ways:

• the MicVAD object passes an empty object as the third parameter to the AudioWorkletNode constructor
• the Processor class found in the worklet.js (and therefore in the worklet bundle) uses a hard-coded value for 'targetFrameSize', instead of using the 'framesamples' parameter
• the MicVAD class uses a custom model fetcher to ensure that the returned model is of type ArrayBuffer
• the AudioNodeVAD class has been altered to ensure that frames to be processed are of type Float32Array (see vadNode.port.onmessage)
• the onnxruntime-web dependency has been updated
• a non-minified worklet bundle file is now generated by the build, which is recommended for Firefox Add-Ons (for reviewing purposes)

Configuration:

• copy the following files into your extension's 'public' directory:

  • ort-wasm-simd-threaded.mjs (available from the onnx-runtime package)
  • vad.worklet.bundle.js (available from this package's /dist directory)
  • silero_vad.onnx (available from this package's root directory)

• you must specify in the options for the MicVAD constructor:

  • the worklet URL, which should point to the worklet bundle file in the public directory
  • an 'ort' configuration that points to your public directory

Quick start:

import { MicVAD, RealTimeVADOptions } from "@jcrawley_dev/firevad";
import { ONNXRuntimeAPI } from "@jcrawley_dev/firevad/dist/_common";

let stream = await navigator.mediaDevices.getUserMedia({
      audio: {
        channelCount: 1,
        echoCancellation: true,
        autoGainControl: true,
        noiseSuppression: true,
      },
    });

const options = {
    workletURL: chrome.runtime.getURL('public/vad.worklet.bundle.js'),
    ortConfig: (ort: ONNXRuntimeAPI) => { 
        ort.env.wasm.wasmPaths = chrome.runtime.getURL('public/');
    },
    positiveSpeechThreshold: 0.8,
    minSpeechFrames: 3,
    preSpeechPadFrames: 10,
    onSpeechStart: () => {
        // your code here
    },
    onSpeechEnd: (rawAudioData: Float32Array) => {
        // your code here
    },
    onVADMisfire: () => {
        // your code here
    },
    onFrameProcessed(probabilities: { isSpeech: number; notSpeech: number }) { 
        // your code here
    },
    stream,
};

let mic = await MicVAD.new(options);
mic.start();

Sponsorship

Please contribute to the original VAD project financially - especially if your commercial product relies on this package.

For more information on the original project: vad.ricky0123.com.