Skip to the content

Route other model providers through the proxy

Send Gemini, Bedrock, Vertex, Azure, OpenRouter, DeepSeek and local-runtime calls through the proxy, accounted, budgeted and checked for secrets.

Built, not yet claimed preview: this guide describes a package that has not passed its release gate.

Point an agent or an app at the proxy's route for its provider, so its model calls are accounted, priced, held under the task's budget and checked for secrets like Anthropic's and OpenAI's, and know which calls a route passes through unaccounted.

1. Know which routes the proxy serves

agentgarm-session's proxy listens on 127.0.0.1:4747 and serves one route per provider endpoint in the provider catalog: /anthropic and /openai; /gemini; /bedrock, /vertex, /foundry and /claude-platform-aws for Claude on the clouds; /azure-openai; /openrouter; /deepseek, /deepseek-anthropic and their -cny twins; /dashscope and /dashscope-anthropic; /volcengine and /volcengine-anthropic; and /local-ollama, /local-vllm, /local-lm-studio and /local-llama-cpp. Any other prefix gets 404.

On every route the proxy refuses subscription logins, checks prompts for secrets, holds each accounted call under the task's budget and prices it from the catalog. A call that names no task is counted against the task of the process that made it. A direct connection to a provider's API host from a governed task is refused, and the proxy answers a WebSocket upgrade with 426, so an agent falls back to HTTPS.

See also

See what each task cost

Route an agent's model calls through AgentGarm so each one is counted and priced, and read the spend by provider and model.

2. Point a Gemini app at the gemini route

agentgarm run and agentgarm wrap always set GOOGLE_GEMINI_BASE_URL to the /gemini route, beside ANTHROPIC_BASE_URL and OPENAI_BASE_URL. Both Google GenAI SDKs read it, so an app built on them needs no change. An OpenAI SDK reaches Gemini through http://127.0.0.1:4747/gemini/v1beta/openai/, set in its own configuration; your key passes through to Google and is never logged.

generateContent, streamGenerateContent and the OpenAI-compatible chat completions are accounted and held. Token counting, embeddings, batches, files and explicit caches pass through unaccounted, and Live, which needs a WebSocket, is refused.

agentgarm wrap -- python3 app.py

See also

Govern an agent with no hooks

Run an agent AgentGarm has no hook adapter for inside the same sandbox and proxy, and know which of your policy's rules that leaves unenforced.

3. Run Claude Code on Bedrock, Vertex, Foundry or Claude Platform on AWS

--route selects a cloud route: agentgarm run sets its base URL and Claude Code's switch (CLAUDE_CODE_USE_BEDROCK, _VERTEX, _FOUNDRY or _ANTHROPIC_AWS) for the launch only. Do not export the switch yourself: an exported routing variable is still replaced inside the launch, but it is reported as a user off-switch and rates the task C. An unknown route refuses the launch.

Foundry and Claude Platform on AWS have no upstream until agentgarm-session is started with --upstream foundry=https://<resource>.services.ai.azure.com/anthropic or --upstream claude-platform-aws=https://aws-external-anthropic.<region>.api.aws, and Bedrock's names its region the same way. On the AWS routes the agent sends its calls unsigned and the session signs them with temporary credentials: the task's brokered ones, or what the session's --aws-credential-export command returns; a long-term access key is refused. On Vertex the sandbox hides ~/.config/gcloud, so export a short-lived access token for the launch.

agentgarm run --route bedrock -- claude
ANTHROPIC_AUTH_TOKEN=$(gcloud auth print-access-token) CLAUDE_CODE_SKIP_VERTEX_AUTH=1 \
  agentgarm run --route vertex -- claude

See also

Give agents short-lived AWS, GCP and SSH credentials

Seal AWS, GCP and SSH credentials in agentgarmd; each task started with agentgarm run gets short-lived ones, and the sealed ones never reach the agent.

4. Point other providers' clients at their routes

Azure OpenAI, OpenRouter, DeepSeek, DashScope and Volcengine Ark get no variable of their own: their vendors document only OPENAI_BASE_URL and ANTHROPIC_BASE_URL, which the first-party routes own. Set the route in the agent's own configuration instead, such as http://127.0.0.1:4747/openrouter/v1 for an OpenAI SDK, or http://127.0.0.1:4747/azure-openai as Codex's Azure base_url. Azure OpenAI needs --upstream azure-openai=https://<resource>.openai.azure.com/openai on the session.

A base URL you exported that names DeepSeek, Volcengine Ark, or an Azure or Foundry resource the session serves is pointed at that provider's route rather than replaced, and one naming a resource the session does not serve refuses the launch, so your key is never sent to another vendor. Like any exported routing variable, it is reported as a user off-switch. The -cny and Volcengine routes bill in yuan: under a cost budget in US dollars their calls are refused, not converted.

See also

Set spend budgets and stop runaway loops

Give each agent task a cost budget and a token rate, see what the agent meets at the cap, and stop loops and runaway subagents with limits of their own.

5. Account for a local runtime

Ollama, vLLM, LM Studio and llama.cpp each have a route to their usual port on loopback. Point the agent at the route, http://127.0.0.1:4747/local-ollama/v1 for Ollama. An exported OPENAI_BASE_URL naming the runtime on its usual port is repointed to its route too, and reported as an off-switch like any other. Inside the sandbox loopback is reachable only through the proxy, so the agent's local calls cannot go around it.

Local calls are counted by tokens and priced at zero. Only chat completions, responses and the model list are forwarded: the runtime's own API (pulling, deleting or loading a model) is refused. A runtime on another port is served with --upstream local-ollama=http://127.0.0.1:<port> on the session; an upstream off loopback is refused, because the route is priced at zero.

See also

Keep agent versions in check

List your installed agents against the versions AgentGarm verified their hooks for, and bring an agent that updated past that range back in range.

6. Check the calls arrive

Open the dashboard with agentgarm ui: Spend lists each call by provider and model, and the task's run shows the model calls it made. A call the catalog cannot price is listed apart as unpriced, and amounts in different currencies are never added. agentgarm status shows each task's assurance level, which reads C after an off-switch was reported.

agentgarm ui
agentgarm status

See also

Check that protection is on

See whether AgentGarm's services are up, what assurance level each agent session has and why, and what to do about a bypass alert.