tui-agent

TUI Agent

Deterministic eyes and hands for OpenTUI applications. Renders a real application into OpenTUI's memory terminal, drives keyboard, paste, mouse, and resize events, and saves text, styled spans, SVG frames, focus state, layout diagnostics, and adapter-owned semantic state after every action.

It is an installable npm package and a Bun CLI. The harness is application-neutral; a small trusted adapter mounts your TUI and optionally exposes routes, dialogs, backend requests, or other state useful for assertions.

Requirements and installation

• Bun 1.2 or newer
• OpenTUI 0.3.4 or newer
• macOS, Linux, or Windows supported by OpenTUI's native packages

bun add --dev tui-agent @opentui/core
bunx tui-agent doctor

The tarball follows normal npm package conventions and can also be installed with npm install --save-dev. The executable requires bun because OpenTUI's test renderer and TypeScript adapter loading use Bun at runtime.

Quick start

Create test/tui/adapter.tsx:

/** @jsxImportSource @opentui/solid */
import { render } from "@opentui/solid"
import type { TuiAgentAdapter } from "tui-agent"

const adapter: TuiAgentAdapter = {
  name: "my-tui",
  async launch({ setup }) {
    await render(() => <App />, setup.renderer)
    return {
      inspect: () => ({ route: "home", state: { ready: true } }),
      dispose() {},
    }
  },
}

export default adapter

Create test/tui/smoke.json:

{
  "$schema": "node_modules/tui-agent/scenario.schema.json",
  "name": "smoke",
  "adapter": "./adapter.tsx",
  "terminal": { "width": 100, "height": 30 },
  "steps": [
    { "action": "waitForText", "text": "Describe the task" },
    { "action": "type", "text": "hello" },
    { "action": "assert", "visible": ["hello"] }
  ]
}

Run it:

bunx tui-agent run test/tui/smoke.json
bunx tui-agent run test/tui/smoke.json --json --artifacts /tmp/tui-smoke

CLI

tui-agent list [--json]
tui-agent run <scenario> [--adapter <module>] [--artifacts <dir>] [--keep-fixture] [--json]
tui-agent interact <scenario> [--adapter <module>] [--artifacts <dir>] [--keep-fixture]
tui-agent show <run-directory> [--json]
tui-agent doctor [--json]

Every command supports --help; failures are written to stderr and return a nonzero exit code.

Scenario model

Scenarios are plain JSON validated against the schema. Available actions:

Action	Purpose
waitForText, waitForAbsent, waitForFocus, waitForIdle	Wait for visual, focus, or idle state
key, type, paste	Keyboard input, modifiers, repetition, and bracketed paste
click, doubleClick, clickText, move, drag, scroll	Mouse input
resize	Trigger real terminal reflow
wait	Pause
capture	Name an evidence frame
find	Search current frame + scrollback buffer for a regex pattern
scrollback	Scroll the in-memory scrollback buffer
assert	Check text, cursor, focus, request evidence, route, dialog, state, and highlights

Each step captures: .txt (text), .svg (styled frame), .spans.json (spans), .tree.json (renderable hierarchy), .state.json (renderer metrics + adapter inspection).

Highlights

Every snapshot includes a highlights field listing background-highlighted spans on screen — the standard way TUI programs indicate selected items, active tabs, or focused buttons. Agents can assert on highlights directly:

{ "action": "assert", "highlights": [{ "text": "Settings" }] }

Adapter API

type TuiAgentAdapter = {
  name: string
  prepare?: (context: AdapterPrepareContext) => AdapterPreparation | Promise<AdapterPreparation>
  launch: (context: AdapterLaunchContext) => RunningTui | Promise<RunningTui>
}

• prepare() — optional setup before scenario runs. Create fixture files, set environment.
• launch() — mount the real TUI on context.setup.renderer. Return dispose + optional inspect.

The inspect() method returns semantic state (route, dialog, mode) that powers declarative assertions without parsing text.

Examples

This repo includes a Spinosa TUI adapter at adapters/spinosa.ts and several scenarios in scenarios/. These demonstrate fixture preparation, renderer factory interception, fake SDK transport, request capture, Effect cleanup, and JSON-safe semantic inspection:

cd packages/tui
bun tools/tui-agent/cli.ts run workspace-session
bun tools/tui-agent/cli.ts interact workspace-session

License

MIT