Cost Breakdown
Get real-time cost information for each API request directly in the response.
LLM Gateway provides real-time cost information for each API request directly in the response's usage object. This allows you to track costs programmatically without needing to query the dashboard.
Cost breakdown is available for all users on both hosted and self-hosted deployments.
Response Format
API responses include cost fields in the usage object:
{
"id": "chatcmpl-123",
"object": "chat.completion",
"created": 1234567890,
"model": "openai/gpt-4o",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! How can I help you today?"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 10,
"completion_tokens": 15,
"total_tokens": 25,
"cost": 0.000125,
"cost_details": {
"upstream_inference_cost": 0.000125,
"upstream_inference_prompt_cost": 0.000025,
"upstream_inference_completions_cost": 0.0001,
"total_cost": 0.000125,
"input_cost": 0.000025,
"output_cost": 0.0001,
"cached_input_cost": 0,
"request_cost": 0,
"web_search_cost": 0,
"image_input_cost": null,
"image_output_cost": null,
"data_storage_cost": 0.00000025
},
"prompt_tokens_details": {
"cached_tokens": 0,
"cache_write_tokens": 0,
"image_tokens": 0,
"audio_tokens": 0,
"video_tokens": 0
},
"completion_tokens_details": {
"reasoning_tokens": 0,
"image_tokens": 0,
"audio_tokens": 0
}
}
}Cost Fields
| Field | Description |
|---|---|
cost | Total inference cost for the request in USD |
cost_details.upstream_inference_cost | Combined upstream inference cost in USD (prompt + completions) |
cost_details.upstream_inference_prompt_cost | Upstream cost for prompt tokens in USD (includes cached prompt discount) |
cost_details.upstream_inference_completions_cost | Upstream cost for completion tokens in USD |
cost_details.total_cost | Total request cost in USD (LLM Gateway extended field) |
cost_details.input_cost | Cost for non-cached prompt tokens in USD |
cost_details.output_cost | Cost for completion tokens in USD |
cost_details.cached_input_cost | Cost for cached prompt tokens in USD |
cost_details.cache_write_input_cost | Cost for prompt tokens written to the provider cache in USD, billed at the provider's cache-write premium (e.g. 1.25x for 5m / 2x for 1h on Anthropic) |
cost_details.request_cost | Per-request flat fee in USD (when the model applies one) |
cost_details.web_search_cost | Cost for web search tool calls in USD |
cost_details.image_input_cost | Cost for image inputs in USD |
cost_details.image_output_cost | Cost for image outputs in USD |
cost_details.data_storage_cost | Storage cost for retained request/response payloads in USD |
Token Detail Fields
The usage object also includes detailed token counters that mirror OpenAI's extended format:
| Field | Description |
|---|---|
prompt_tokens_details.cached_tokens | Number of prompt tokens served from the provider's prompt cache |
prompt_tokens_details.cache_write_tokens | Number of prompt tokens written into the provider's prompt cache |
prompt_tokens_details.cache_creation_tokens | Alias of cache_write_tokens, matching Anthropic's naming |
prompt_tokens_details.cache_creation | Per-TTL breakdown of cache writes (ephemeral_5m_input_tokens / ephemeral_1h_input_tokens), present when a cache write occurred |
prompt_tokens_details.image_tokens | Number of image prompt tokens, as reported by the provider |
prompt_tokens_details.audio_tokens | Number of audio prompt tokens |
prompt_tokens_details.video_tokens | Number of video prompt tokens |
completion_tokens_details.reasoning_tokens | Number of reasoning tokens produced by reasoning models |
completion_tokens_details.image_tokens | Number of image tokens produced |
completion_tokens_details.audio_tokens | Number of audio tokens produced |
Streaming Responses
Cost information is also available in streaming responses. The cost fields are included in the final usage chunk sent before the [DONE] message:
data: {"id":"chatcmpl-123","object":"chat.completion.chunk","choices":[...],"usage":{"prompt_tokens":10,"completion_tokens":15,"total_tokens":25,"cost":0.000125,"cost_details":{"upstream_inference_cost":0.000125,"upstream_inference_prompt_cost":0.000025,"upstream_inference_completions_cost":0.0001,"total_cost":0.000125,"input_cost":0.000025,"output_cost":0.0001,"cached_input_cost":0,"request_cost":0,"web_search_cost":0,"image_input_cost":null,"image_output_cost":null,"data_storage_cost":0.00000025}}}
data: [DONE]Every Endpoint
Every billed reply carries usage.cost, what the request was billed in USD, with usage.cost_details breaking it down; cost_details.total_cost equals cost. There is nothing to opt into.
| Endpoint | Where the cost is | What it was billed on, in usage |
|---|---|---|
/v1/chat/completions | usage; on streams the final chunk before [DONE] | prompt_tokens, completion_tokens, total_tokens, and the token details above |
/v1/messages | usage; on streams message_delta.usage | input_tokens (not cached), output_tokens, cache_read_input_tokens, cache_creation_input_tokens, cache_creation, server_tool_use |
/v1/responses | usage; on streams response.usage in response.completed | input_tokens, output_tokens, total_tokens, input_tokens_details.cached_tokens, output_tokens_details.reasoning_tokens |
/v1/responses/compact | usage | as /v1/responses |
/v1/images/generations, /v1/images/edits | usage | input_tokens, output_tokens, total_tokens, input_tokens_details and output_tokens_details (image_tokens, text_tokens) |
/v1/embeddings | usage | prompt_tokens, total_tokens |
/v1/decisions | usage | input_tokens, output_tokens, total_tokens |
/v1/rerank | usage | input_tokens, total_tokens when the model is priced per token |
/v1/audio/transcriptions | usage | type: "duration", seconds of input audio |
/v1/ocr | usage | pages_processed |
/v1/moderations, /v1/search | usage | billed per request: cost_details.request_cost |
/v1/audio/speech | with stream_format: "sse", usage of the final speech.audio.done event | input_tokens, output_tokens, total_tokens for models priced per token, input_characters for models priced per character |
/v1/videos | usage of GET /v1/videos/{id} and the signed callback, once the job is billed | cost_details.video_output_cost, cost_details.image_input_cost |
Speech returned as binary audio (the default) has no body to carry a cost. Request stream_format: "sse" to receive the audio as speech.audio.delta events followed by speech.audio.done with its usage and cost; this works for every speech model.
A video job reports usage once it reached a terminal status and was billed: a job that failed, was canceled or expired reports a cost of 0. If a terminal job has no usage yet, poll again.
Example: Tracking Costs in Code
Here's an example of how to track costs programmatically using the cost breakdown feature:
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.LLM_GATEWAY_API_KEY,
baseURL: "https://api.llmgateway.io/v1",
});
async function trackCosts() {
const response = await client.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: "Hello!" }],
});
const usage = response.usage as any;
if (usage.cost !== undefined) {
console.log(`Request cost: $${usage.cost.toFixed(6)}`);
console.log(
` Prompt: $${usage.cost_details.upstream_inference_prompt_cost.toFixed(6)}`,
);
console.log(
` Completions: $${usage.cost_details.upstream_inference_completions_cost.toFixed(6)}`,
);
const cachedTokens = usage.prompt_tokens_details?.cached_tokens ?? 0;
if (cachedTokens > 0) {
console.log(` Cached prompt tokens: ${cachedTokens}`);
}
}
return response;
}Use Cases
Budget Monitoring
Track costs in real-time and implement budget limits in your application:
let totalSpent = 0;
const BUDGET_LIMIT = 10.0; // $10 budget
async function makeRequest(messages: Message[]) {
const response = await client.chat.completions.create({
model: "gpt-4o",
messages,
});
const cost = (response.usage as any).cost || 0;
totalSpent += cost;
if (totalSpent > BUDGET_LIMIT) {
throw new Error(`Budget exceeded: $${totalSpent.toFixed(2)}`);
}
return response;
}Per-User Cost Allocation
Track costs per user for billing or analytics:
const userCosts: Map<string, number> = new Map();
async function makeRequestForUser(userId: string, messages: Message[]) {
const response = await client.chat.completions.create({
model: "gpt-4o",
messages,
});
const cost = (response.usage as any).cost || 0;
const currentCost = userCosts.get(userId) || 0;
userCosts.set(userId, currentCost + cost);
return response;
}Cost Analytics
Aggregate costs by model, time period, or any other dimension:
interface CostEntry {
timestamp: Date;
model: string;
promptCost: number;
completionsCost: number;
totalCost: number;
}
const costLog: CostEntry[] = [];
async function loggedRequest(model: string, messages: Message[]) {
const response = await client.chat.completions.create({
model,
messages,
});
const usage = response.usage as any;
costLog.push({
timestamp: new Date(),
model: response.model,
promptCost: usage.cost_details?.upstream_inference_prompt_cost || 0,
completionsCost:
usage.cost_details?.upstream_inference_completions_cost || 0,
totalCost: usage.cost || 0,
});
return response;
}Self-Hosted Deployments
If you're running a self-hosted LLM Gateway deployment, cost breakdown is always included in API responses regardless of plan. This allows you to track internal costs and allocate them across teams or projects.
How is this guide?
Last updated on