Docs › LLM Gateway

LLM Gateway (OpenAI-compatible /v1)

Payment wires: every paid endpoint accepts x402 and MPP (Machine Payments Protocol) on the same 402 - see Paying with x402 and Paying with MPP. Agent402 is the applied layer of Agentic Finance: agents that pay and get paid on their own.

Acceptable use. Hosted-instance traffic is governed by the Terms of Service - including a generative-content acceptable-use policy - and by the upstream model providers' usage policies. Wallets used for prohibited content are blocked before settlement. Outputs are generated by third-party models from your inputs; you are responsible for how you use them.

Pay-per-call inference and embeddings at the OpenAI wire paths - any OpenAI SDK, agent framework, or plain HTTP client adopts the gateway by changing one setting:

A plain OpenAI SDK cannot sign a payment, so give it one of two payers. The simplest is the local wallet proxy, which signs an x402 payment per call from a wallet you hold and forwards to the gateway:

npm i @x402/fetch @x402/evm viem
export AGENT402_WALLET_KEY=0x...   # an EVM key holding USDC on Base
npx agent402-openclaw proxy        # serves http://127.0.0.1:8412/v1

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8412/v1", api_key="unused")
client.chat.completions.create(model="auto", messages=[{"role": "user", "content": "hi"}])

The same proxy answers the Anthropic wire (POST /v1/messages), so Claude Code pays from the wallet too: ANTHROPIC_BASE_URL=http://127.0.0.1:8412.

Or pay in-process from JavaScript by wrapping fetch with an x402 payer (wrapFetchWithPayment from @x402/fetch) and handing it to the SDK's fetch option; agent402-client does the same with spend caps built in.

Ordering, because it decides who eats an upstream failure: the handler runs first, and @x402/express settles afterwards, and only for a response below 400. A 502 from a provider therefore cancels settlement and you are not charged; the failover chains below exist so a provider outage becomes a retry rather than your 502. A 200 is charged once settlement succeeds, and streaming starts only after that (see Streaming below). Upstream is OpenRouter for chat, images and speech, and OpenAI for embeddings; per-tier model allowlists and input/output caps bound what one call can ask for. GET /v1/models (free) lists every model with its tier, price, and caps.

Because tiers are flat-priced while upstream bills per token, every request is also priced server-side before it goes upstream: input tokens are counted exactly (including tool schemas and images), and max_tokens is automatically tightened when an expensive model plus a large input would otherwise approach the tier price. Cheap and mid-priced models never hit this bound. A request whose input alone exceeds the budget returns a 400 explaining the fix (shrink the input, lower n, or pick a cheaper model). n is capped at 4.

Privacy routing: send zdr: true (or provider: {zdr: true}) on any chat tier to route only to providers with a zero-data-retention policy. Same price, same models where available; a model with no ZDR provider errors upstream and the failover chain walks to one that has it. zdr is the only provider preference a buyer can set - pricing bounds stay server-owned.

Tiers

Endpoint Price Serves Input cap Output cap
POST /v1/nano/chat/completions $0.003 nano models (gpt-6-luna by default, gpt-5.6-luna, gpt-5-nano, gemini flash-lite, small llama/ministral/qwen, deepseek-chat, laguna) - priced for high-frequency agent loops 12k chars 768 tokens
POST /v1/auto/chat/completions $0.01 model optional - deterministic eval-ranked routing (see below) 16k chars 1,024 tokens
POST /v1/chat/completions $0.02 budget/mid models (gpt-4o-mini, gpt-4.1-mini, claude haiku, gemini flash-lite, deepseek, llama, mistral, qwen) 32k chars 2,048 tokens
POST /v1/pro/chat/completions $0.10 mid-frontier (gpt-4o, gpt-4.1, gpt-6 sol, claude sonnet incl. sonnet-5, gemini pro and 3.x flash, grok) 48k chars 4,096 tokens
POST /v1/premium/chat/completions $0.50 frontier (gpt-5 and gpt-6 astra, o3 and o4-mini, claude opus incl. opus-5.5, and claude fable 5.1) 85k chars 8,192 tokens
POST /v1/embeddings $0.002 text-embedding-3-small (default), 3-large, ada-002 - batch up to 64 inputs 16k chars -
POST /v1/images/generations $0.08 FLUX.2 Pro with GPT-5 Image Mini as the failover - one 1024x1024 PNG per call, inline base64 out 4k-char prompt 1 image
POST /v1/audio/speech $0.06 a five-model failover chain on OpenRouter's audio API; raw mp3/pcm bytes out, the 11 OpenAI voice names plus each model's native voices 2k-char input -

Bare OpenAI-style names (gpt-4o-mini) are accepted and mapped; requesting a model that another flat tier serves is priced at that tier: the 402 quotes its price, the paid call is served under its caps and failover, and the answer says so in agent402_tier. All tiers are wallet-only - every call burns real upstream credit, so there is no proof-of-work free tier (see Security Model). The pro and premium chat tiers also accept three upstream server tools under a server-owned bound (openrouter:web_search, openrouter:web_fetch, openrouter:datetime, each with a hard use cap that GET /v1/models lists per tier); other server tools are refused by name.

Metered tier - pay per request, quoted first

POST /v1/metered/chat/completions (v1-chat-metered) serves every explicit model the flat chat tiers serve (the same allowlists; GET /v1/models names it beside each chat model as meteredEndpoint, with meteredMaxInputChars 200k and meteredMaxTokens 8,192), but the price is not flat: the 402 quotes this exact request from its own body. The quote is the request's worst case (input counted exactly, plus your max_tokens, at the model's list price) times a fixed markup, never below the $0.001 settlement floor, rounded up to a micro-dollar; a request whose quote would exceed $2 is refused with a 400 naming the cap (a 4xx cancels settlement, so nothing is charged). Because the price is derived from the body actually served, a payment authorized for a small quote cannot ride a bigger request. Set max_tokens to what you need: it is what you pay for.

Two ways to pay it:

  • Exact clients (most stock x402 clients) pay the quote. A short call costs a fraction of a cent; a long one pays for what it asks.
  • upto clients (an x402 client with the upto scheme registered, which on Base needs a one-time USDC approval to Permit2) authorize the quote as a ceiling and settle actual usage under it. Prepaid-credits buyers get the same treatment automatically: the quote is held, and only actual usage is debited on a 200.

The catalog lists the tier at its $0.001 floor ("from"), and every chat model on GET /v1/models carries meteredFromUsd. Use the flat tiers when you want a known price per call regardless of length; use the metered tier when calls vary a lot in size and you want to pay for the size you send.

A flat-tier answer tells you when metered was cheaper. An OpenAI SDK puts base_url + /chat/completions on the flat route by convention, so a buyer who never reads the docs can spend a long time on a $0.02 call that the metered route would have quoted at $0.001. A non-streaming flat-tier response therefore carries an additive agent402_metered object whenever the same body would have been at least 25% cheaper metered:

"agent402_metered": {
  "endpoint": "/v1/metered/chat/completions",
  "wouldHaveCostUsd": 0.001,
  "youPaidUsd": 0.02,
  "note": "The same request on the metered route is quoted from its own body and settles actual usage. Point your SDK's base URL at /v1/metered to use it."
}

The field is absent when metered would not be materially cheaper, on the metered tier itself, and on streamed responses (a stream has no envelope to carry it). It is not part of the documented response shape, so nothing should depend on its presence.

OpenClaw provider plugin (agent402-openclaw)

OpenClaw talks to any OpenAI-compatible provider through one block in openclaw.json, and the agent402-openclaw npm plugin writes that block for this gateway: AGENT402_CREDITS_KEY=a402_... npx agent402-openclaw setup --write stores a prepaid credits key already issued (new credits are not on sale at the moment; see /credits) and starts a loopback proxy that carries it, so OpenClaw itself never holds a payment credential. auto (routed per prompt, flat $0.01 per call) is offered beside every id on GET /v1/models; explicit models ride the metered route by default (--flat keeps them on their flat tiers), and setup picks the cheapest preferred metered model whose input cap holds OpenClaw's own system prompt as the primary. A wallet can pay instead of a credits key (AGENT402_WALLET_KEY, an EVM key holding USDC on Base): exact by default, or upto after a one-time agent402-openclaw permit2-approve, so the wallet settles actual usage; agent402-openclaw doctor reports which mode it is in. Full guide: agent402.tools/guides/openclaw-model-provider.

Image generation

POST /v1/images/generations speaks the OpenAI images wire - any OpenAI SDK's images.generate() works by changing base_url. Send {"prompt": "..."} (up to 4,000 chars) and get {created, model, data: [{b64_json, media_type}]} back - one image per call at a flat $0.08, n locked to 1, response_format is always inline b64_json (nothing is hosted). zdr: true works here too. Upstream is FLUX.2 Pro (GPT-5 Image Mini as the failover) on OpenRouter's Image API, each pinned to one provider with a per-image bound and a live price re-check.

Text-to-speech

POST /v1/audio/speech speaks the OpenAI TTS wire, so any OpenAI SDK's audio.speech.create() works by changing base_url. Send {"input": "...", "voice": "alloy"} (up to 2,000 chars) and raw audio bytes come back, mp3 by default and pcm on request, at a flat $0.06 per call. zdr: true works here too.

There is no OpenAI TTS model on the upstream, so the tier serves a five-model failover chain over OpenRouter's audio API instead, walking to the next link on any upstream failure including an empty audio body. The 11 OpenAI voice names (alloy, ash, ballad, coral, echo, fable, onyx, nova, sage, shimmer, verse) are accepted on every link and mapped to that model's closest native voice; each model's native voice ids (e.g. en_paul_cheerful) are also accepted directly and are listed per model on GET /v1/models. Pinning model moves that link to the front of the chain rather than disabling the rest.

Two request fields differ from OpenAI's:

  • instructions is rejected with a self-explaining 400: no serving model supports it. Pick an expressive native voice instead.
  • speed is accepted anywhere in 0.25–4; most serving models ignore it.

The auto tier - routing without picking a model

Send messages with no model field (or model: "auto") and the gateway routes the prompt to the top-ranked model for its task type. Routing is fully deterministic - lexical classification (code / reasoning / long-context / general) against a fixed, eval-derived ranking table; no LLM in the routing path, identical requests always route identically.

An optional quality field picks the ranking band at the same flat price (a per-request price cannot exist under x402's fixed per-route quote):

  • "fast" - cheapest/snappiest serving, right for loop turns
  • "balanced" - the default
  • "best" - the strongest models the price still covers

The response discloses the decision alongside the standard model field:

"agent402_router": { "category": "code", "quality": "balanced", "served": "deepseek/deepseek-chat" }

Each ranking doubles as a failover chain: if a provider returns 502/503/504, the gateway walks down the list instead of charging you for an upstream error - every chain ends in the model the daily paid canary proves alive.

Streaming

Add stream: true on any chat tier for standard OpenAI SSE framing (data: {chunk} … data: [DONE]). Payment settles before the first byte; max_tokens is clamped server-side so the stream stops at the tier cap. Streamed responses are not idempotency-replayable and never cached.

Response caching

Two policies over one cache (10-minute TTL, served before the paywall - a hit costs nothing):

  • Chat tiers - opt-in. LLM output is sampled, and a resend usually wants a fresh sample, so nothing is cached unless the request carries cache: true. A byte-identical opted-in repeat that carries your payment returns the stored response with X-Cache: hit, and the payment is not charged (a Tempo transfer sent before the call is final when sent, so it counts); without a payment the request gets the usual 402.
  • Embeddings - default-on. Embeddings are deterministic per model, so an identical repeat is free automatically: it carries your payment as usual and the payment is not charged (a Tempo transfer sent before the call is final when sent, so it counts). Opt out with cache: false.

Keys are computed over the normalized body (model aliases and field order collapse; every sampling-relevant field is included), so equivalent requests share one entry.

  • POST /api/route/execute ($0.01; execute-plus $0.05, execute-max $0.55 and execute-pro $3.30 for pricier tools) - resolve a task description to the best-matching tool and run it in the same call, returning {result, receipt}. With include:"external" it can buy the result from a proven external x402 seller and sell it to you. See x402 Index and Router.
  • POST /api/my-usage ($0.005) - your wallet's own purchase history (totals, per-tool counts, per-chain breakdown, receipts with settle txs). No wallet parameter: the x402 payment that buys the report determines whose report it is - nobody can read another wallet's profile.
  • The older custom-JSON proxies remain available: LLM Proxy Gateway (/api/llm*) and Text Embeddings (/api/embed*).

Verified daily

The paid canary buys from the gateway every day with real USDC: a nano completion (exercising the failover chain), a model-less auto completion (asserting the router disclosure), a live SSE stream, an embeddings vector, and both cache behaviors (paid once → identical repeat served free). If any of it breaks, an alarm issue opens on the repo.

Upstream service tiers

Where the upstream offers it (Gemini 2.5/3.x families, gpt-5-nano, gpt-5.6-*, and the image model), the gateway asks for OpenRouter's flex service tier first (higher latency, lower availability) and retries the same model on the default tier if flex has no capacity, before moving to the next failover link. Buyers see the same price either way; the response's service_tier field says which tier served.

Prompt caching

Every chat call asks the upstream to cache the prompt prefix (OpenRouter's top-level cache_control: {type:"ephemeral"}, 5-minute TTL) and pins your turns to one provider (session_id), so a multi-turn agent conversation is served from the provider cache on repeated prefixes. Same flat price to you either way. Send cache_control: false to opt out. The budget tiers (nano, auto) additionally pick the cheapest provider under their price cap; pro and premium keep OpenRouter's default provider balancing.

Rerank

POST /v1/rerank ($0.002) speaks the Cohere rerank wire: {query, documents[], top_n} in, results[{index, relevance_score, document}] out, served by cohere/rerank-v3.5. Up to 50 documents (1,600 chars each, 40k total) and a 500-char query per call. Deterministic, so a byte-identical repeat within 10 minutes is served free from cache (cache:false opts out). Pair it with /v1/embeddings: embed, recall your top candidates, rerank them.

Anthropic Messages API

The same five tiers also speak the Anthropic Messages wire: POST /v1/nano/messages, /v1/auto/messages, /v1/messages, /v1/pro/messages, /v1/premium/messages (same prices, models, caps and failover as each tier's chat route). Point the Anthropic SDK, Claude Code or the Agent SDK at https://agent402.tools/v1 (or /v1/pro, /v1/premium) with an x402-paying fetch and call messages.create as usual: system, content blocks (text, image, tool_use, tool_result), client tools, thinking, stop_sequences, streaming (message_start … message_stop). Any model on the tier is served through this wire (Claude natively, others translated upstream). max_tokens is required by the wire and clamped to the tier cap; a reply that spends its whole cap thinking and says nothing is never served (the failover chain walks on, and an exhausted chain is a 502 you are not charged for). On Claude 4.7+ (Opus 5, Sonnet 5, Fable 5.1) the depth control is top-level effort (low, medium, high, xhigh, max), validated against the model's own list from /v1/models and refused with that list; older Claude takes thinking.budget_tokens instead. speed other than standard is refused: the priority endpoint bills twice the model's list price and sits outside every tier's price.

OpenAI Responses API

The same five tiers also speak the OpenAI Responses wire: POST /v1/nano/responses, /v1/auto/responses, /v1/responses, /v1/pro/responses, /v1/premium/responses. Point responses.create() (or the OpenAI Agents SDK) at https://agent402.tools/v1 (or a tier prefix) through an x402-paying fetch. input as a string or item list, instructions, function tools, text.format, reasoning, streaming (response.created … response.completed) all work; server-side tools (web search, file search, computer use, MCP) are not served, and there is no stored conversation state (store is always false, previous_response_id is refused) - send the full input each call.

POST /v1/grounded/chat/completions ($0.03) is the auto router plus a live Exa web search on every call (up to 5 results): the model answers from the results and the reply carries url_citation annotations. Omit model (or send auto) and the gateway picks the model by task type; the response adds agent402_router. :online model variants are refused on the other tiers (pro and premium accept the bounded openrouter:web_search server tool instead), and here search is part of the flat price. Never cached.

Other hosts (Claude Code, Cursor, Continue, ElizaOS, AgentCore, any OpenAI SDK): https://agent402.tools/guides/agent-hosts