0.0.3 • Published yesterdayCLI

@wuhezhizhong/local-ai-gateway

Licence

MIT

Version

0.0.3

Deps

Size

399 kB

Vulns

Weekly

Summary Dependency Versions

Local AI Gateway

Local lightweight routing gateway for AI clients and custom model providers.

Configuration and secrets

gateway.config.json contains only public routing configuration. Every clients.<client>.endpoints[] entry has a stable, read-only ID such as ep_550e8400-e29b-41d4-a716-446655440000.

Credentials are stored separately in ignored gateway.secrets.json:

{
  "api_keys": {
    "ep_550e8400-e29b-41d4-a716-446655440000": "env:ARK_API_KEY",
    "ep_7c8b91e1-43cd-4dc7-bd13-7ca32a511cee": "sk-local-secret"
  }
}

On first start, legacy providers, models, official_models, missing endpoint IDs, and endpoint api_key fields are migrated automatically. The original config is backed up as gateway.config.json.<timestamp>.bak.

Each client can have one is_default fallback endpoint. expose_models is independent: when one or more endpoints opt in, only their models are listed; when none opt in, every endpoint is listed. Public model IDs (the values in models and keys in model_mapping) must be unique within a client. Duplicate IDs are rejected with endpoint-name-based candidate names.

Claude Desktop / Claude Code
  -> http://127.0.0.1:8787/v1/messages
  -> Anthropic-compatible providers
  -> OpenAI Chat-compatible providers

Codex
  -> http://127.0.0.1:8787/codex/v1/responses
  -> OpenAI-compatible providers

OpenAI-compatible clients
  -> http://127.0.0.1:8787/v1/chat/completions
  -> OpenAI Chat, OpenAI Responses, or Anthropic-compatible providers

The gateway started as a Volcengine Ark bridge, but the main configuration is now provider-based. You can add Volcengine, OpenRouter, DeepSeek, Qwen, OpenAI, Anthropic, or any compatible provider by editing gateway.config.json. Claude Code can also use mixed routing: mapped models go to Ark, while unmapped real Claude model IDs are forwarded to the official Anthropic Messages endpoint.

Files

server.js: local gateway server
.env.example: config template
docs/providers.md: provider recipes for Volcengine, OpenRouter, DeepSeek, and Anthropic
scripts/gateway.mjs: cross-platform background gateway control
scripts/init-config.mjs: cross-platform first-run config initializer
tests/: unit, integration, and isolated E2E tests

Provider Config

New setups should create the local environment file:

npm run init

The init command creates .env and gateway.config.json from public templates only when the target files do not already exist, so it is safe to run again. Local .env and gateway.secrets.json are ignored by Git; gateway.config.json is safe to commit because endpoint credentials are stored separately. The published npm package intentionally excludes the repository's configured gateway.config.json and installs an empty endpoint template for each user.

Start the gateway, open http://127.0.0.1:8787/config, add endpoints, and save the page to create gateway.config.json.

The web config stores endpoints under each client:

{
  "server": { "host": "127.0.0.1", "port": 8787 },
  "clients": {
    "code": { "endpoints": [] },
    "desktop": { "endpoints": [] },
    "codex": { "endpoints": [] }
  }
}

Provider type currently supports:

anthropic         -> /v1/messages providers
openai-chat       -> /v1/chat/completions providers
openai-responses  -> /v1/responses providers

For configured third-party providers, base_url is the exact upstream request URL. The gateway does not append /messages, /chat/completions, or /responses to it.

openai-chat providers can also serve Codex /v1/responses requests through the gateway's Responses-to-Chat adapter. This is useful for providers such as OpenRouter, DeepSeek, Qwen, Moonshot, and other OpenAI Chat-compatible APIs.

The same openai-chat providers can serve Claude Desktop / Claude Code /v1/messages requests through the gateway's Anthropic-to-Chat adapter. That means a Claude-style client model id such as claude-sonnet-openrouter can be mapped to an upstream OpenAI-compatible model such as deepseek/deepseek-chat.

Start

npm run init

Edit .env:

# Optional. If empty, the gateway uses the client's Gateway API key as Ark key.
ARK_API_KEY=your-volcengine-ark-api-key
GATEWAY_HOST=127.0.0.1
GATEWAY_PORT=8787
GATEWAY_CONFIG_FILE=gateway.config.json
ARK_BASE_URL=https://ark.cn-beijing.volces.com/api/plan
ARK_CODEX_BASE_URL=https://ark.cn-beijing.volces.com/api/plan/v3
OFFICIAL_ANTHROPIC_BASE_URL=https://api.anthropic.com

Then run:

npm start

Open http://127.0.0.1:8787/config and save the web config page to create gateway.config.json.

For background control on Windows, macOS, or Linux, use:

npm run gateway:start
npm run gateway:status
npm run gateway:stop
npm run gateway:restart

Background mode writes:

gateway.pid.json
gateway.stdout.log
gateway.stderr.log
gateway.log

Use the unified command directly when you need an isolated port or runtime directory:

npm run gateway -- start --test
npm run gateway -- status --test
npm run gateway -- stop --test

Test mode always uses port 8788, writes runtime files under .gateway-test, and disables Claude Desktop, Claude Code, and Codex configuration sync.

Desktop App

The desktop app is an Electron shell for the same web config page served at /config. It starts the gateway when the app opens, loads the local http://127.0.0.1:<port>/config page, and stops the gateway when the app exits. It does not keep a separate desktop config UI.

Run it during development:

npm install
npm run desktop

Build installers:

npm run desktop:dist

The build config emits a Windows NSIS installer on Windows and a macOS dmg on macOS. macOS signing/notarization is not configured.

Platform-specific build commands are also available:

npm run desktop:dist:win
npm run desktop:dist:mac

The repository also includes a GitHub Actions workflow at .github/workflows/desktop-build.yml that builds both platforms on native hosted runners and uploads the Windows and macOS artifacts.

The desktop app keeps its local .env, gateway.config.json, and logs in Electron's user data directory, not in the repository root. The config is still created by saving the shared web config page.

Desktop checks:

npm run desktop:check
npm run desktop:test
npm run desktop:smoke

desktop:smoke expects a Windows unpacked build in dist/win-unpacked. It starts the packaged app on a temporary high port, checks /health, exits the app, and verifies that the port is released.

Validate

After saving gateway.config.json, validate your provider config:

npm run validate:config

Run adapter tests without calling any real upstream provider:

npm run check
npm run test:adapters

Run the local doctor when a client cannot connect or a model does not appear:

npm run doctor

Global Command

After installing the package globally, the same cross-platform control is available without npm:

npm install -g @wuhezhizhong/local-ai-gateway
local-ai-gateway start
local-ai-gateway status
local-ai-gateway stop
local-ai-gateway restart
local-ai-gateway logs

The first command creates user-owned configuration under ~/.local-ai-gateway/. Existing .env and gateway.config.json files are never overwritten.

Claude Desktop/Gateway Config

Gateway base URL: http://127.0.0.1:8787/desktop
Gateway auth scheme: bearer
Gateway API key: your Volcengine Ark API key, unless ARK_API_KEY is set in .env
Model discovery: enabled

Claude Code Config

Use a different base URL so logs can distinguish Claude Code from Claude Desktop:

http://127.0.0.1:8787/code

For pure Volcengine third-party mode, keep ANTHROPIC_AUTH_TOKEN set to your Ark key in ~/.claude/settings.json.

For Claude Code subscription pass-through plus custom Ark models, set only the base URL and remove ANTHROPIC_AUTH_TOKEN / ANTHROPIC_API_KEY from the Claude Code env block:

{
  "env": {
    "ANTHROPIC_BASE_URL": "http://127.0.0.1:8787/code",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "claude-sonnet-4-5",
    "ANTHROPIC_DEFAULT_SONNET_MODEL_NAME": "glm-5.2"
  },
  "model": "sonnet"
}

Routing behavior:

claude-sonnet-4-5       -> deepseek-v4-pro on Ark, from gateway.config.json
glm-5.2                 -> glm-5.2 on Ark, from gateway.config.json alias
claude-sonnet-4-6       -> official Anthropic upstream
claude-haiku-4-5-...    -> official Anthropic upstream

Both prefixed and unprefixed routes work:

http://127.0.0.1:8787/v1/messages          -> client: unknown or inferred
http://127.0.0.1:8787/desktop/v1/messages  -> client: desktop
http://127.0.0.1:8787/code/v1/messages     -> client: code

The gateway also accepts x-gateway-client: desktop or x-gateway-client: code, but URL prefixes are the most reliable.

Codex Desktop Config

Use the Codex-prefixed base URL:

http://127.0.0.1:8787/codex

Current OpenAI-style routes exposed for Codex testing:

http://127.0.0.1:8787/codex/v1/models
http://127.0.0.1:8787/codex/v1/chat/completions
http://127.0.0.1:8787/codex/v1/responses
http://127.0.0.1:8787/codex/v1/config
http://127.0.0.1:8787/codex/v1/providers
http://127.0.0.1:8787/codex/health

These Codex routes are forwarded through configured OpenAI-compatible providers, with model mapping handled locally by this gateway.

Supported Codex provider matrix:

Codex upstream	Text	Image	Reasoning	Tools
Official subscription	Native	Native	Native	Native
OpenAI Responses	Native	Capability-based	Native	Native
OpenAI Chat	Adapted	Capability-based	Adapted summary	Adapted
Grok Responses	Native	Capability-based	Native	Native
Grok Chat	Adapted	Capability-based	Adapted summary	Adapted

GET /codex/v1/models merges official models with configured third-party IDs. Official discovery starts from the local Codex bundled catalog, then optionally refreshes that bundled list and adds any extra gpt-* / o* IDs from https://api.openai.com/v1/models when local Codex auth is available. Live fetch failures fall back to bundled data and never change request routing. Disable live refresh with CODEX_MODELS_LIVE_DISABLED=1.

Generate or refresh the Desktop model catalog:

npm run codex:catalog
npm run codex:catalog:verify

Both the gateway (on start/save) and npm run codex:catalog write the same default file:

~/.codex/gateway-model-catalog.json

Override with CODEX_MODEL_CATALOG_PATH if you need a project-local file. codex:catalog:verify checks that codex debug models -c ... can see the custom models and prints an optional config.toml snippet. It does not edit Codex's user config by itself. If you test the snippet manually, insert it before the first [section] so model_provider and model_catalog_json remain top-level TOML keys.

Codex isolated verification

Run npm run test:codex:e2e before editing ~/.codex/config.toml. The harness uses a temporary Codex home, temporary fixture, local mock provider, and a temporary gateway port.

After it passes, back up ~/.codex/config.toml, add the generated local-gateway provider snippet, and verify one official subscription model before selecting a third-party model. Roll back by restoring the backup and restarting Codex Desktop; ~/.codex/auth.json is not modified.

OpenAI-Compatible Client Config

For clients that speak OpenAI Chat Completions directly, use the unprefixed base URL:

http://127.0.0.1:8787

The gateway exposes:

http://127.0.0.1:8787/v1/models
http://127.0.0.1:8787/v1/chat/completions
http://127.0.0.1:8787/v1/responses
http://127.0.0.1:8787/v1/config
http://127.0.0.1:8787/v1/providers
http://127.0.0.1:8787/v1/resolve?model=glm-5.2

Use any local model id from gateway.config.json; the gateway maps it to the provider's upstream_model. Chat Completions requests can be forwarded directly to openai-chat providers, or translated to openai-responses and anthropic providers when the selected model uses one of those upstream types.

/v1/resolve?model=... is a safe local debug endpoint. It shows the selected provider, upstream model, and whether Claude Messages, OpenAI Chat, and OpenAI Responses routes are direct, translated, official, or unsupported.

Test

Invoke-RestMethod http://127.0.0.1:8787/health
Invoke-RestMethod http://127.0.0.1:8787/v1/models
Invoke-RestMethod http://127.0.0.1:8787/v1/config

Messages test:

$body = @{
  model = "claude-sonnet-4-5"
  max_tokens = 256
  messages = @(
    @{ role = "user"; content = "hello" }
  )
} | ConvertTo-Json -Depth 10

Invoke-RestMethod `
  -Uri http://127.0.0.1:8787/v1/messages `
  -Method Post `
  -ContentType "application/json" `
  -Body $body

Keywords

ai gateway claude codex anthropic openai