@aituber-onair/comment-intelligence

AITuber OnAir

日本語版はこちら

Build stream-ready AI VTubers with TypeScript

AITuber OnAir is an open source toolkit for building AI VTubers that can chat, speak, react to viewers, use memory, and run with PNG, VRM, or Live2D avatars. Start from the hosted web app, scaffold a starter app, self-host a working example, or assemble your own stack from modular TypeScript packages.

Try the hosted web app ・ Quickstart ・ Examples ・ Packages

What you can build

• AI VTubers that chat and speak with live viewers
• Streaming assistants that react to YouTube / Twitch comments
• AI character apps with text, voice, vision, and long-term memory
• Viewer relationship systems with points, levels, and achievements
• Browser- and Node.js-based integrations, composed from independent packages

Start in 10 minutes

If you want the shortest path to a local AI VTuber:

npm create aituber-onair@latest my-aituber
cd my-aituber
npm run dev

Then open the app, choose a template, and configure your LLM / TTS provider from Settings. See the Quickstart for the full walkthrough.

Choose your path

1. Try the hosted web app

AITuber OnAir is a full, standalone AITuber streaming web app built on top of @aituber-onair/core. It's both the quickest way to experience the toolkit end-to-end and a working reference for what you can ship with it. No setup required.

2. Create a starter app

Use create-aituber-onair to scaffold your own app from an official PNGTuber, VRM, or Live2D starter template.

npm create aituber-onair@latest

The CLI asks for a project name, template, and whether to install dependencies. You can also pass the project name up front:

npm create aituber-onair@latest my-aituber
cd my-aituber
npm run dev

For step-by-step setup and template selection, see Quickstart.

3. Run an example app locally

Four full, ready-to-run React apps built on @aituber-onair/core. Pick the avatar style that fits your project. All four share the same broad LLM / TTS provider coverage and in-app Settings UI.

PNGTuber Chat — 2D PNG avatar

Swap in 4 PNG states (mouth/eyes open/close) and get real-time lip-sync driven from actual audio output. See packages/core/examples/react-pngtuber-app.

git clone https://github.com/shinshin86/aituber-onair.git
cd aituber-onair/packages/core/examples/react-pngtuber-app
npm install
npm run dev

VRM Chat — 3D VRM avatar

Render a 3D VRM avatar (miko.vrm) with optional idle VRMA animation, real-time mouth lip-sync driven from audio output, and camera controls (drag to rotate / wheel to zoom). See packages/core/examples/react-vrm-app.

git clone https://github.com/shinshin86/aituber-onair.git
cd aituber-onair/packages/core/examples/react-vrm-app
npm install
npm run dev

Live2D Chat — local Live2D folder loader

Live2D sample model: Hiyori Momose. Illustration: Kani Biimu; Modeling: Live2D Inc. This content uses sample data owned and copyrighted by Live2D Inc. See Live2D Sample Data.

Load a local Live2D model folder that contains .model3.json, render it in the browser, and drive mouth movement from actual audio output volume. This example intentionally does not bundle any Live2D assets. See packages/core/examples/react-live2d-app.

git clone https://github.com/shinshin86/aituber-onair.git
cd aituber-onair/packages/core/examples/react-live2d-app
npm install
npm run dev

Pet Chat — animated Codex Pet-style sprite

Render a Codex Pet-compatible spritesheet, move it around the stage, and switch animations from chat state, reply mood, and real-time audio volume. See packages/core/examples/react-pet-app.

git clone https://github.com/shinshin86/aituber-onair.git
cd aituber-onair/packages/core/examples/react-pet-app
npm install
npm run dev

Open http://localhost:5173 in any case, then set API keys and provider options in Settings.

See Examples for the full example map and recommended starting points.

4. Build your own with the packages

Install only what you need and drop it into your own app:

npm install @aituber-onair/chat

import { ChatServiceFactory } from '@aituber-onair/chat';

const chat = ChatServiceFactory.createChatService('openai', {
  apiKey: process.env.OPENAI_API_KEY!,
});

await chat.processChat(
  [{ role: 'user', content: 'Hello!' }],
  (partial) => process.stdout.write(partial),
  async (full) => console.log('\nDone:', full),
);

See each package README for provider setup and fuller usage.

Documentation

• Quickstart: create a starter app, pick a template, configure providers, and run locally.
• Examples: choose from full AI VTuber apps, package examples, bot examples, and local runtime examples.

Packages

create-aituber-onair

CLI for creating an AITuber OnAir app from an official starter template. Currently includes PNGTuber, VRM, Live2D, and Pet templates. PNGTuber, VRM, and Pet include starter assets; Live2D does not bundle Live2D model assets.

npm create aituber-onair@latest

@aituber-onair/core

Core runtime tying chat, voice, memory, and conversation context together for full AITuber experiences.

npm install @aituber-onair/core

@aituber-onair/chat

Unified LLM layer across OpenAI, Claude, Gemini, Z.ai, Kimi, DeepSeek, Mistral, and OpenRouter — streaming, tool/function calling, vision, and MCP support included.

npm install @aituber-onair/chat

@aituber-onair/voice

Standalone TTS library with VOICEVOX, VoicePeak, OpenAI TTS, MiniMax, AIVIS Speech, and more, plus emotion-aware synthesis.

npm install @aituber-onair/voice

@aituber-onair/manneri

Detects repetitive conversation patterns and injects topic-diversification prompts to keep dialogue fresh.

npm install @aituber-onair/manneri

@aituber-onair/noise

Post-generation rewrite engine that keeps AI character replies from landing too safely: detects predictable phrasing, builds structured friction, asks an LLM for rewrite candidates, and selects the safest non-generic response.

npm install @aituber-onair/noise

@aituber-onair/comment-intelligence

Filters live comments before they reach an AI character: selects one comment to answer, blocks unsafe input, summarizes ignored comments, and builds compact LLM context. Rules-first with optional LLM-assisted analysis.

npm install @aituber-onair/comment-intelligence

@aituber-onair/bushitsu-client

WebSocket chat client with React hooks, auto-reconnect, rate limiting, mentions, and voice integration. Browser and Node.js.

npm install @aituber-onair/bushitsu-client

@aituber-onair/kizuna

Relationship / bond system (絆) for AI characters and viewers: points, achievements, emotion-based bonuses, level progression, persistent storage.

npm install @aituber-onair/kizuna

Why AITuber OnAir

• Proven in production — powers AITuber OnAir, a live AITuber streaming web app, so you're building on the same code path a real product ships on
• Pick any entry point: hosted web app, starter CLI, self-hosted example, or modular npm packages
• First-class coverage of the providers AITuber builders actually use — OpenAI / Claude / Gemini for chat, VOICEVOX / OpenAI TTS / AIVIS Speech and more for voice
• Chat, voice, streaming (YouTube / Twitch / WebSocket), and viewer relationships in a single, consistent stack
• MIT-licensed TypeScript — you keep control of hosting, data, and integrations

Project structure

aituber-onair/
└── packages/
    ├── create-aituber-onair/ # npm create CLI with starter templates
    ├── core/             # AITuberOnAirCore, memory, orchestration
    ├── chat/             # LLM providers, streaming, tools, MCP
    ├── voice/            # TTS engines, emotion, playback
    ├── manneri/          # Conversation pattern detection
    ├── noise/            # Post-generation response rewriting
    ├── comment-intelligence/ # Live comment filtering and context building
    ├── bushitsu-client/  # WebSocket chat client + React hooks
    └── kizuna/           # Viewer relationship / bond system

License

MIT — see LICENSE.

Special Thanks

This project is based on the work referenced here. Without the contributions of these pioneers, it would not exist.

---

For contributors

Working on the monorepo itself:

git clone https://github.com/shinshin86/aituber-onair.git
cd aituber-onair
npm install
npm run build
npm run test
npm run fmt

Agent Skills

Shared Agent Skills so Codex and Claude Code use the same workflow definitions. See docs/agent-skills.md for the full guide. Canonical sources live in skills/, with Claude Code runtime copies under .claude/skills/.

Releases

Releases are driven by manual version bumps + per-package CHANGELOG.md, published automatically by GitHub Actions on merge to main. Do not run npm publish directly.

• Patch: bug fixes, dependency updates
• Minor: new features, backward-compatible changes
• Major: breaking changes to public API

release.yml uses Changesets to publish packages, create tags (@aituber-onair/<pkg>@x.y.z), and create GitHub Releases for packages published in that run. If CI fails mid-run, re-running publishes the remainder but does not backfill Releases for already-published packages — create those manually from the package CHANGELOG (tag will already exist). prerelease-next.yml only updates the next prerelease tag.