Service Tiers

Some OpenAI, Google, and Fireworks models support selectable processing tiers that trade latency and availability against price. You pick one per request with the OpenAI-compatible service_tier parameter, and LLM Gateway forwards it only when the selected provider/model mapping supports that tier.

Using the service_tier parameter

curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google-vertex/gemini-3.1-pro-preview",
    "service_tier": "priority",
    "messages": [
      { "role": "user", "content": "Summarize this incident report." }
    ]
  }'

Accepted values are flex, priority, and default/auto (standard). If you request flex or priority for a provider/model mapping that does not support that tier, the gateway returns a 400 unsupported_service_tier error and logs the request as a client error.

The parameter works the same on the OpenAI-compatible Responses API (/v1/responses): the tier is forwarded to the provider and the response's service_tier field echoes the tier that was actually served.

Coding (dev) plans are limited to default/auto and flex — the premium priority tier is not part of the plan and a request that asks for it returns a 403.

Supported providers

Service tiers are explicit per provider/model mapping. Check the model page for the exact tiers exposed by each provider card.

• OpenAI (openai) — sent as the OpenAI service_tier request field for supported OpenAI models. Flex is billed at 0.5x standard token prices and Priority uses the model-specific multiplier shown on the model page.

• Google Vertex AI (google-vertex) — sent as the X-Vertex-AI-LLM-Shared-Request-Type request header, together with X-Vertex-AI-LLM-Request-Type: shared so the request bypasses any Provisioned Throughput on the project and actually reaches the shared Flex/Priority tier. Flex and Priority are served only on the global endpoint, which is the gateway default. Google Flex PayGo applies a 0.5x multiplier; Google Priority PayGo applies a 1.8x multiplier.

• Google AI Studio / Gemini API (google-ai-studio) — sent as a service_tier field in the request body for configured models that opt in.

• Fireworks AI (fireworks) — sent as a service_tier field in the request body on its OpenAI-compatible chat completions endpoint. Only Priority is offered (Fireworks publishes no Flex rate card), billed at a 1.25x multiplier. Fireworks does not report the tier it served, but a Priority request is either served at Priority or shed with a 503, so an accepted request is billed at the tier it was sent at.

Retries and fallback never downgrade the tier

A request can change provider or credential mid-flight: LLM Gateway falls back to another provider when one fails, and rotates to another key for the same provider when a credential returns a 429 or an auth error. A requested tier is carried through all of it.

• Provider routing is narrowed to mappings that support the requested tier before a provider is picked, so no fallback candidate can be one that would serve the request as standard.
• Key selection — BYOK keys, platform-managed credentials, and env credentials alike — skips any credential that cannot carry the tier (a proxy base URL, or a Vertex credential pinned to a regional endpoint), on the first attempt and on every retry.
• Every attempt re-resolves the tier against the provider, region and credential it actually resolved to, and fails rather than sending at a lower tier. If no eligible candidate is left, you get the upstream error instead of a silently downgraded response.

This applies to a tier you requested yourself. The optional coding-plan default tier is a cost preference rather than a requirement, so a request that cannot be served at that tier runs at standard instead of failing; used_service_tier in the response metadata always reports what was actually served. A tier sent on the request always takes precedence over that default, within the tiers the plan allows.

Pricing uses multipliers

Service tiers do not define separate model prices in LLM Gateway. They multiply the provider mapping's standard token prices:

• Standard / default / auto: 1x
• Flex: 0.5x
• Priority: model/provider-specific, shown on the model page

The multiplier scales per-token costs, including input, output, cached, and image tokens. Flat per-request and web-search fees are not tier-scaled.

Billing follows the served tier

When a provider reports the tier that was actually served, LLM Gateway bills that returned tier instead of blindly billing the requested value:

• A priority request that runs as priority is billed at 2.5x.
• A flex request that runs as flex is billed at 0.5x.
• A request that is served as standard is billed at the standard 1x rate.

The served tier is read back from the provider response — Vertex reports it in usageMetadata.trafficType (ON_DEMAND_PRIORITY / ON_DEMAND_FLEX / ON_DEMAND), Google AI Studio reports it in the x-gemini-service-tier response header, and OpenAI can return service_tier in response payloads or stream events. Providers that report no tier at all (Fireworks) never downgrade silently — they reject the request instead — so an accepted request is billed at the tier it was sent at.

You can see per-tier pricing for each model on its model page. Supported provider cards include a Service Tier selector in the card header and show the active multiplier next to each tier.

Sources

• OpenAI API pricing
• Google Flex PayGo
• Google Priority PayGo
• Fireworks Serverless Priority and Fast