peito-denoise

Peito Denoise

English · Русский

Real-time AI noise suppression for the browser, Electron and Capacitor WebView. Neural noise removal runs entirely on the client, end-to-end latency ~60 ms, shipped as the npm library peito-denoise.

Demo: https://peito.ru

What it is

The library takes any Web Audio source (microphone, MediaStreamTrack, file) and returns cleaned audio in real time — with no inference server, the whole model runs in the browser via onnxruntime-web. The core is a TypeScript port of the DeepFilterNet3 DSP pipeline plus streaming neural inference. Around the core is the commercial layer: landing page, user dashboard, admin panel, offline ECDSA-signed licenses, online activation and payment via YooKassa.

A particular focus of the project is observability. A denoiser easily "eats" speech or introduces musical artifacts, and that is hard to catch by ear. So the package ships side-by-side input/output spectrograms, live metrics, distortion detectors and raw/processed WAV export for offline reproduction.

Features

• Real-time in-browser inference. DeepFilterNet3 via onnxruntime-web (wasm), streaming frame-by-frame with GRU state carried across frames. Inference p50 ~4 ms/frame — faster than real time.
• Multiple backends out of the box. dfn3 (neural net), spectral (pure TS: STFT + Wiener gain, ~24 dB on stationary noise, no assets) and passthrough. Switch at runtime without rebuilding the audio graph.
• Pluggable backends. Plug in your own model (RNNoise / DTLN / GTCRN / any ONNX) via registerBackend without touching the pipeline — a per-frame processing interface with internal state.
• Observability. Metrics batch at ~15 Hz: in/out RMS and peak in dB, actual reductionDb, noise floor, inferMs p50/p95/p99, deadlineMisses, ring fill, under/overruns, measured latency, VAD. Input/output spectrograms and WAV dump.
• Distortion detectors. clip, click/gap, musical, oversuppress (model eats speech while VAD is active), nan/inf (guard with frame mute), dead — emitted as events with a ring-buffered history log.
• Fail-open watchdog. If the worker goes silent longer than watchdogMs, the denoiser automatically passes raw audio through (not silence) and recovers when the worker revives. All wasm/worker boundaries are wrapped; errors go to an event, never silently.
• AutoGain. Brings quiet speech up to a target level: VAD gate, per-sample smoothing, lookahead limiter — no clicks or clipping, with a bounded boost.
• Licensing. Offline verification via ECDSA P-256 signature (WebCrypto): domains with wildcards, term, features. The primary path is online activation with encrypted model delivery and instant revocation.
• Cross-platform. Browser, Electron, Capacitor WebView; a TrackProcessor adapter for LiveKit.

Tech stack

Core / DSP-ML TypeScript, Web Audio API (AudioWorklet), Web Workers, SharedArrayBuffer + Atomics, onnxruntime-web, DeepFilterNet3 (ONNX, streaming split), custom DSP (STFT/ISTFT, ERB filterbank, Bluestein FFT, biquads, VAD, AutoGain).

Storefront backend Node.js + TypeScript (tsx), custom HTTP layer, JSON store with atomic writes, ECDSA key signing (WebCrypto), YooKassa (checkout + webhook with API-side status verification), nodemailer (email confirmation, password reset), Redis / in-memory rate limiting and sessions.

Infrastructure Vite (library and site builds), Docker Compose, Caddy (static + automatic Let's Encrypt TLS + proxy), Redis, Selectel S3 (encrypted model delivery via presigned URLs), Prometheus + Grafana, encrypted DB backups (AES-256-GCM).

Architecture

The hot audio path is moved off the main thread and built so it never blocks. The AudioWorklet upholds the real-time contract: its process() does no inference and no allocations — only reads and writes to ring buffers, and on underrun it fails open to raw audio. All heavy computation lives in a Web Worker: model inference, metrics, distortion detectors, ring recording for the dump.

Exchange between worklet and worker goes through a lock-free SPSC ring buffer on SharedArrayBuffer plus a separate block of atomics for control and counters. postMessage is used only for rare control messages and a throttled metrics batch — it is never on the hot path. To fight jitter and clicks, the output ring is primed with silence (headroom for GC and bursty consumption by the audio device), and excess input backlog is drained in rare large chunks rather than frequent small drops.

The DeepFilterNet3 DSP core is ported from upstream libDF to TypeScript: STFT 960/480 with a Vorbis window, a 32-band ERB filterbank, normalizations and ISTFT. Port correctness is verified by round-trip reconstruction — error of 2.4e-7. The 960-point FFT (not a power of two) is computed via the Bluestein algorithm. Inference mirrors upstream 1 enc_conv → enc_gru(h) → erb_dec(h) + df_dec(h), ERB mask application and deep filter (5 taps × 96 bins) with GRU states carried across frames.

SharedArrayBuffer requires cross-origin isolation, so all serving runs under COOP same-origin / COEP require-corp headers, and models from S3 are served with Cross-Origin-Resource-Policy: cross-origin. On a real domain this requires HTTPS (secure context) — in the deployment, Caddy issues and renews the TLS certificate.

mic / track / file → MediaStreamSource → AudioWorklet (realtime ring IO)
                                              |
                     SharedArrayBuffer: lock-free SPSC ring + atomics
                                              |
                          Web Worker (model inference + metrics + detectors)
                                              |
                     AudioWorklet out → destination / MediaStreamDestination

Numbers

Metric	Value
End-to-end latency	~60 ms (~30 ms algorithmic: 2-frame lookahead + hop; 20–40 ms anti-jitter buffer)
DFN3 inference	p50 ~4 ms/frame (wasm), faster than real time
DSP core round-trip error	2.4e-7
Spectral denoiser	~24 dB on stationary noise, no assets
ONNX model size	≈8.5 MB (streaming split, 4 models)
Rate / frame (spectral)	48 kHz, hop 256, FFT 512, 50% overlap, sqrt-Hann
Activation API throughput	~3300 activations/s (single-node benchmark)

What ships beyond the core

• npm library peito-denoise with types, worklet and a separate peito-denoise/panel entry point (debug panel with metrics, detector lamps and a log — not in the production bundle).
• Storefront: landing page with pricing, integration docs, denoiser demo bench.
• User dashboard: sign-up and sign-in with email confirmation and password recovery, license list, purchase (free plan instantly, paid via YooKassa), domain changes with key reissue, revocation.
• Admin panel: live sessions, payments, licenses, revenue, uptime; auto-refresh.
• One-command deploy: Docker Compose with a Caddy + API + Redis stack + encrypted backups + Prometheus/Grafana, model delivery from S3.

Reliability is a priority: atomic DB writes (temp + rename + .prev) with auto-recovery from .prev, graceful degradation when Redis is down (fail-open to in-memory, activations do not break), graceful shutdown on SIGTERM, container health checks and auto-restart, client-side activation retries with exponential backoff, encrypted backups every 15 minutes.

Status

Commercial product. Source code is closed; access for technical review is available on request. Live demo at the link above.