Anthropic API Compatibility

LLMGateway provides a native Anthropic-compatible endpoint at /v1/messages that allows you to use any model in our catalog while maintaining the familiar Anthropic API format This is especially useful for applications designed for Claude that you want to extend to use other models.

Overview

The Anthropic endpoint transforms requests from Anthropic's message format to the OpenAI-compatible format used by LLMGateway, then transforms the responses back to Anthropic's format. This means you can:

• Use any model available in LLMGateway with Anthropic's API format
• Maintain existing code that uses Anthropic's SDK or API format
• Access models from OpenAI, Google, xAI, and other providers through the Anthropic interface
• Leverage LLMGateway's routing, caching, and cost optimization features

Basic Usage

Configuration for Claude Code

This endpoint is perfect for configuring Claude Code to use any model available in LLMGateway:

export ANTHROPIC_BASE_URL=https://api.llmgateway.io
export ANTHROPIC_AUTH_TOKEN=llmgtwy_your_api_key_here
# optional: specify a model, otherwise it uses the default Claude model
export ANTHROPIC_MODEL=gpt-5  # or any model from our catalog

# now run claude!
claude

Choosing Models

You can use any model from the models page. Popular options for Claude Code include:

# Use OpenAI's latest model
export ANTHROPIC_MODEL=gpt-5

# Use a cost-effective alternative
export ANTHROPIC_MODEL=gpt-5-mini

# Use Google's Gemini
export ANTHROPIC_MODEL=gemini-3.1-pro-preview

# Use Anthropic's actual Claude models
export ANTHROPIC_MODEL=claude-sonnet-4-6

Environment Variables

When configuring Claude Code or other Anthropic-compatible applications, you can use these environment variables:

ANTHROPIC_MODEL

Specifies the main model to use for primary requests.

• Default: claude-sonnet-4-20250514
• Example: export ANTHROPIC_MODEL=gpt-5

ANTHROPIC_SMALL_FAST_MODEL

Specifies a smaller, faster model used for background functionality and internal operations.

• Default: claude-3-5-haiku-20241022
• Example: export ANTHROPIC_SMALL_FAST_MODEL=gpt-5-nano

# Example configuration
export ANTHROPIC_BASE_URL=https://api.llmgateway.io
export ANTHROPIC_AUTH_TOKEN=llmgtwy_your_api_key_here
export ANTHROPIC_MODEL=gpt-5
export ANTHROPIC_SMALL_FAST_MODEL=gpt-5-nano

Advanced Features

Making a manual request

curl -X POST "https://api.llmgateway.io/v1/messages" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5",
    "messages": [
      {"role": "user", "content": "Hello, how are you?"}
    ],
    "max_tokens": 100
  }'

Response Format

The endpoint returns responses in Anthropic's message format:

{
	"id": "msg_abc123",
	"type": "message",
	"role": "assistant",
	"model": "gpt-5",
	"content": [
		{
			"type": "text",
			"text": "Hello! I'm doing well, thank you for asking. How can I help you today?"
		}
	],
	"stop_reason": "end_turn",
	"stop_sequence": null,
	"usage": {
		"input_tokens": 13,
		"output_tokens": 20
	},
	"metadata": {
		"requested_model": "gpt-5",
		"used_model": "gpt-5",
		"used_provider": "openai"
	}
}

metadata is the same gateway object /v1/chat/completions returns: it names the provider, model and region that served the request, and lists every attempt in routing when a fallback happened (see Routing Transparency). On streaming responses it is attached to the message_delta event.

Request Format

/v1/messages expects Anthropic Messages requests and always answers in Anthropic's format. Because the two formats share model and messages, an OpenAI Chat Completions body can reach this endpoint by accident — and since unknown parameters are ignored rather than rejected, the request succeeds and returns an Anthropic response body that OpenAI SDKs cannot read. If your client reports an empty completion here, check that it is pointed at /v1/chat/completions.

Unknown parameters are deliberately ignored rather than rejected, so a valid Anthropic request is never denied for carrying an extra field. As a consequence, OpenAI-only parameters (response_format, stream_options, max_completion_tokens, n, stop, seed, frequency_penalty, and similar) have no effect here — the model will not honour them. Use /v1/chat/completions if you need them.

A body that is structurally OpenAI is rejected by the schema, as it always has been — OpenAI-shaped tools ({"type": "function", "function": {…}}), OpenAI content parts such as image_url, or assistant turns with content: null. Those rejections now name the mismatch and point at the right endpoint instead of returning an opaque validation error:

{
	"type": "error",
	"error": {
		"type": "invalid_request_error",
		"message": "This endpoint implements Anthropic's Messages API, and the request body uses OpenAI Chat Completions structures (tools[0].function) that Anthropic's format has no equivalent for. Send OpenAI-format requests to /v1/chat/completions instead, or convert the body to Anthropic's Messages format."
	}
}

Rejected requests are recorded in your logs with a client_error finish reason and zero cost, so a malformed client is visible in the activity feed rather than failing silently.

Prompt Caching

For Claude models, cache_control markers on system and message content blocks are forwarded to the provider unchanged, including the optional ttl (5m or 1h):

{
	"model": "claude-sonnet-4-6",
	"max_tokens": 100,
	"system": [
		{
			"type": "text",
			"text": "<several thousand tokens of stable instructions...>",
			"cache_control": { "type": "ephemeral" }
		}
	],
	"messages": [{ "role": "user", "content": "Hello!" }]
}

Cache usage comes back in Anthropic's native fields: usage.cache_creation_input_tokens (tokens written to the cache this request, billed at the write premium), usage.cache_read_input_tokens (tokens served from cache at the discounted rate), and usage.cache_creation (the per-TTL write breakdown).

A system message inside messages (as Claude Code sends on recent Claude models) stays at its position instead of being merged into the top-level system prompt, so the cached conversation prefix is not rewritten on every turn. Where the upstream model does not accept that role, or does not accept it at that position (it must follow a user turn and precede an assistant turn or end the conversation), it is sent in place as a user message wrapped in <system-reminder> tags. If system messages interrupt a complete tool-call/result turn, the gateway moves them after all results for that turn so the results remain adjacent to the tool calls. Incomplete turns and unrelated user messages are left unchanged. Claude Code's tool_addition and tool_removal blocks are accepted and passed to the model as a text note.

Tool choice

tool_choice (auto, any, none, or {"type": "tool", "name": …}) is honoured on every provider: it is translated into the upstream's own forced-tool form (Anthropic any/tool, OpenAI required/named function, Gemini functionCallingConfig), and a mapping that cannot force a tool is skipped during routing. disable_parallel_tool_use: true is carried as well; on providers without an equivalent it is ignored. An OpenAI-style tool_choice value ("required", {"type": "function", …}) is dropped rather than rejected.

Tool definitions and results

A tool's strict: true reaches the Anthropic API (with its structured-outputs beta) and OpenAI-compatible providers as function.strict; providers without an equivalent receive the tool without it. allowed_callers is forwarded to the Anthropic API only, with the advanced-tool-use beta, and stripped elsewhere.

Gemini models take tool schemas through Google's typed parameters carrier, which has no const or additionalProperties. A const is sent as a single-member enum (typed from the value when the node has no type; on non-string nodes, where Vertex rejects enum, it becomes an "Allowed values" hint in the description), and an explicit enum on the same node wins. pattern is forwarded on string nodes. additionalProperties is dropped. The same translation applies on /v1/chat/completions.

An image block inside a tool_result (a screenshot returned by a tool, say) is delivered to Anthropic models as an image block; a result that mixes text and images keeps its order. Other providers receive it as the chat-completions image part their tool message format allows, or as text where there is none.

Web Search

Anthropic's server-side web search tool works on this endpoint. Pass it as usual and the response carries server_tool_use and web_search_tool_result blocks before the text that cites them, so Anthropic SDK clients surface sources:

{
	"model": "claude-haiku-4-5",
	"max_tokens": 400,
	"messages": [{ "role": "user", "content": "What shipped in Node 24?" }],
	"tools": [{ "type": "web_search_20250305", "name": "web_search" }]
}

Replaying the assistant turn verbatim on the next request is supported: the server_tool_use and web_search_tool_result blocks are accepted and dropped, since the provider re-runs the search rather than reusing the previous results.

Tool Search

Anthropic's server-side tool search works on this endpoint. Pass a tool_search_tool_* tool alongside your catalog and mark the tools that should load on demand with defer_loading: true:

{
	"model": "claude-sonnet-4-6",
	"max_tokens": 1024,
	"messages": [{ "role": "user", "content": "What is the weather in Paris?" }],
	"tools": [
		{
			"type": "tool_search_tool_regex_20251119",
			"name": "tool_search_tool_regex"
		},
		{
			"name": "get_weather",
			"description": "Get the weather at a specific location",
			"input_schema": { "type": "object" },
			"defer_loading": true
		}
	]
}

Deferred tools stay out of the rendered tools section, so adding one does not invalidate an existing prompt cache. The response carries the server_tool_use and tool_search_tool_result blocks; replay them verbatim on the next request and Anthropic keeps expanding the tool_reference entries they carry, so Claude reuses a discovered tool instead of searching again. tool_reference blocks returned from your own client-side search inside a tool_result are forwarded unchanged too.

Send every tool definition on every request, including the deferred ones — Anthropic needs them server-side to run the search. At least one tool must stay non-deferred (normally the tool search tool itself), and a tool cannot carry both defer_loading: true and cache_control.

Where it works. Tool search reaches the provider on the Anthropic API and on Anthropic models served through Google Cloud, and it needs a Claude 4.5-generation model or newer — older Claude models reject it upstream. On every other provider, including Anthropic models on AWS Bedrock, the tool search tool and defer_loading are dropped and all tools are sent eagerly. The request still succeeds, it just loses the cache and token savings, so pin the provider (anthropic/claude-sonnet-4-6) when those savings matter.

Auto Mode Safeguard Review

In auto mode, Claude Code asks Anthropic to review risky tool calls as part of its normal requests, at no charge: it sends a safeguards field with a paired anthropic-beta value, and Anthropic returns the verdicts in safeguard_results (on the final message_delta event when streaming). The gateway forwards both halves to the Anthropic API and returns the verdicts unchanged, so Claude Code keeps using the free server-side checks instead of its own billed classifier requests. Run /status in Claude Code to confirm: the Auto mode server row reads Enabled.

Routing requirements. A nonempty safeguards field paired with its dated beta restricts automatic selection and retries to providers that support forwarding server-side review. Currently this is the Anthropic API. An incompatible saved session pin is replaced; an explicitly requested incompatible provider, or automatic selection with no compatible provider, returns HTTP 400, including for streaming requests. A beta header alone does not restrict routing.

Upstream availability still depends on the model, credentials, and rollout. If Anthropic returns no available verdict, Claude Code can fall back to its own billed classifier requests. To use another provider deliberately, set CLAUDE_CODE_AUTO_MODE_SERVER=0 before starting Claude Code; its client-side classifier requests are billed normally.

Gateway Response Cache

If gateway caching is enabled on the project, a repeated request with the same cache-key fields — resolved provider and model, messages, and the other keyed parameters; fields outside the cache key and insignificant JSON whitespace don't affect matching, but the key order inside message and tool objects does, and a request routed to a different provider is a cache miss — is replayed from cache instead of being sent upstream. The replayed body keeps the original id, content, and token counts, so check the x-llmgateway-cache: HIT response header or metadata.cached. Send x-no-cache: true to bypass the cache for a single request.