llm-observer

LLM Observer

Privacy-first, local-only LLM cost tracking for developers.

Stop sending your prompt data to SaaS observability tools. LLM Observer runs entirely on your machine — auto-detects usage from Claude Code, Cursor, Aider, GitHub Copilot, Windsurf, Cline, and OpenAI Codex CLI, or proxies live OpenAI/Anthropic/Gemini/Mistral/Groq/Ollama calls — calculates costs, and visualises everything in a real-time dashboard at localhost:4001.

Your API keys, prompts, and responses never leave your machine unless you explicitly opt into a feature that sends less than that elsewhere (see Data & privacy).

---

Quick start

npx llm-observer start

That's it. Proxy starts on port 4000, dashboard on port 4001. On first launch it also auto-detects and parses session history from any supported editor already on your machine — no proxy setup required for that part.

---

How it works

Session tracking (Claude Code, Cursor, Aider, Copilot, Windsurf, Cline, Codex CLI) needs zero setup — it just reads files those tools already write.

For live traffic — full per-request prompt/response capture, real-time cost, and budget kill switches — point your existing LLM code at the local proxy instead of the provider directly:

OpenAI (Node.js)

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: 'your-actual-key',           // still goes here, stored locally
  baseURL: 'http://localhost:4000/v1/openai',
});

Anthropic (Node.js)

import Anthropic from '@anthropic-ai/sdk';

const anthropic = new Anthropic({
  apiKey: 'your-actual-key',
  baseURL: 'http://localhost:4000/v1/anthropic',
});

Google Gemini

baseURL: 'http://localhost:4000/v1/google'

Mistral / Groq

baseURL: 'http://localhost:4000/v1/mistral'
baseURL: 'http://localhost:4000/v1/groq'

Ollama (local, first-class — always tracked at $0 cost)

baseURL: 'http://localhost:4000/v1/ollama'

Any other OpenAI-compatible endpoint (LM Studio, self-hosted, etc.)

baseURL: 'http://localhost:4000/v1/custom/http%3A%2F%2Flocalhost%3A1234'

Every proxied request is intercepted, logged, costed, and shown in the dashboard — zero changes to your application logic.

---

Features

100% Private by default

All data stored in a local SQLite database at ~/.llm-observer/data.db. No telemetry. No third-party servers. Your prompts and API keys never leave your machine unless you opt into the proxy (still local-only storage) or the AI Analyst (sends only aggregated numbers, never prompts, and only with your own API key).

Real-time dashboard

Live cost counter, request log with filters, latency tracking, model breakdown charts, cost trajectory, subagent trees, statistical A/B comparison, and a reasoning-chain debugger — all at http://localhost:4001.

Budget guard

Set a daily budget per project. When spend hits the limit, the proxy can hard-block new requests before you wake up to a surprise bill.

llm-observer budget set 5.00 --daily

Anomaly & drift detection

Spend-spike alerts via webhook (Slack, Discord, or any HTTP endpoint), plus opt-in response-drift detection that flags when a project's outputs start statistically diverging from its own baseline.

Cost optimizer + AI Analyst

Identifies duplicate prompts and cheaper model alternatives with real savings estimates. Optionally, bring your own Anthropic key and get a Claude-generated summary of your spend — built only from aggregated metadata, never your prompts.

7 providers supported

OpenAI · Anthropic · Google Gemini · Mistral · Groq · Ollama (first-class, $0 cost) · Custom/Local (LM Studio or any other OpenAI-compatible endpoint)

Opt-in PII redaction

Regex + Luhn-validated detection for emails, phone numbers, SSNs, credit cards, and API keys/tokens in proxied traffic — off by default, one toggle in Settings.

100+ models priced

Pricing database covering GPT-5.x, Claude 5/Opus 4.8/Sonnet 5/Haiku 4.5, Gemini 3.x, Grok 4.x, DeepSeek V4, Mistral, and more — kept current, see the root repo's pricing.json.

---

CLI commands

llm-observer start                   # Start proxy + dashboard
llm-observer stop                    # Stop all services
llm-observer status                  # Check if the proxy and dashboard are online

llm-observer stats                   # Cost breakdown (--model / --provider filters)
llm-observer logs --tail             # Live tail of requests
llm-observer logs --limit 50         # Show the last N requests

llm-observer projects                # List all projects and budget statuses
llm-observer budget set 10.00 --daily --project <id>

llm-observer export --format csv --range 30d   # Export last 30 days
llm-observer export --format json --range all

llm-observer pricing update          # Refresh pricing from the remote registry
llm-observer audit --range 30d       # Enterprise audit log export
llm-observer config                  # Show current settings (read-only for now)
llm-observer reset --force           # Wipe the local database

License activation and upgrading to Pro currently happen in the Dashboard (Settings → License & Billing tab), not the CLI. llm-observer activate <key> and llm-observer upgrade exist but are placeholders — a follow-up will wire them to the same backend the dashboard already uses correctly.

---

Dashboard pages

Page	What it shows
Control Room (Overview)	Today's spend vs budget, request count, avg latency, error rate, cost trend
Requests	Every request in real-time via SSE — provider, model, tokens, cost, latency, status, and a reasoning-chain view for any single request
Optimize	Health score, optimization rules, ROI/plan-value, AI Analyst recommendations
Compare	Statistical A/B comparison between models, projects, or time windows
Insights	Cost optimizer suggestions, duplicate prompt detection, model downgrade opportunities
Projects	Multi-project cost isolation — separate budgets per app or environment
Alerts	Webhook alert rules for budget thresholds and anomaly spikes
Settings	API key management, proxy config, PII redaction, drift detection, license activation

---

Pricing

Plan	Price	Features
Free	$0 forever	1 project · 7-day log retention · Budget guard · Anomaly alerts
Pro	$19/mo	Unlimited projects · 90-day retention · Cost optimizer · CSV/PDF export · Priority support
Pro (India)	₹1,599/mo	Same as Pro, billed via Razorpay
Team	$49/seat/mo	Everything in Pro + encrypted team sync + shared dashboard (team-server auth backend exists; dashboard/CLI integration is still in progress — see the main repo README)

Purchase via the website (card or UPI/Razorpay) — your license key is emailed automatically, then activated in Dashboard → Settings → License & Billing.

---

Why not Helicone, Langfuse, or LangSmith?

	LLM Observer	Helicone	Langfuse	LangSmith
Data stays local	Always	Cloud	Cloud	Cloud
No account required
Works offline
Your prompts exposed	Never	To their servers	To their servers	To their servers
Free tier	Unlimited local	Limited	Limited	Limited

If you're working on anything sensitive — client data, proprietary prompts, internal tooling — LLM Observer is the only observability tool where your data genuinely never leaves your machine.

---

Requirements

• Node.js 18+
• macOS / Linux / Windows

---

Configuration

llm-observer config               # Print current API URL and ports (read-only)

Set via environment variables (see the main repo README for the full, current list):

LLM_OBSERVER_PORT=4001 LLM_OBSERVER_PROXY_PORT=4000 npx llm-observer start

# PROXY_PORT / DASHBOARD_PORT also still work, kept for backward compatibility

---

Data & privacy

• All data stored locally at ~/.llm-observer/data.db (SQLite)
• Free tier: 7-day automatic log retention
• Pro tier: 90-day retention
• To delete all data: llm-observer reset --force (or rm ~/.llm-observer/data.db)
• Zero telemetry, zero analytics, zero outbound connections except: your configured LLM provider, a background npm-registry check for newer versions (package name only, disable with NO_UPDATE_NOTIFIER=1), and — only if you opt in — the AI Analyst (aggregated metadata only) and license validation (a license key, nothing else)

---

Links

• GitHub
• Issues
• npm

---

License

MIT Ranjit Behera