Realtime API

LLMGateway supports low-latency, speech-to-speech conversations through the OpenAI-compatible /v1/realtime WebSocket endpoint. Sessions support text and audio input/output, server-side voice activity detection (VAD), input audio transcription, and function calling — using the same event protocol as the OpenAI Realtime API. The same endpoint also serves transcription-only sessions for live speech-to-text.

The API is a drop-in replacement: point any OpenAI realtime client at wss://api.llmgateway.io/v1/realtime, authenticate with your LLMGateway API key, and keep your existing event handling.

Available Models

Browse the available realtime models, with up-to-date pricing, on the models page. Realtime sessions are billed per token — text and audio input, cached input, and output are metered separately at the model's listed rates, matching the provider's own pricing.

Model names work like everywhere else on LLMGateway: use the plain model id (e.g. gpt-realtime), a dated alias, or the provider/model pinned form (e.g. openai/gpt-realtime).

Connecting

Connect a WebSocket to:

wss://api.llmgateway.io/v1/realtime?model=gpt-realtime

Authenticate with your LLMGateway API key in the Authorization header (an x-api-key header also works):

import WebSocket from "ws";

const url = "wss://api.llmgateway.io/v1/realtime?model=gpt-realtime";
const ws = new WebSocket(url, {
	headers: {
		Authorization: "Bearer " + process.env.LLM_GATEWAY_API_KEY,
	},
});

ws.on("open", () => {
	console.log("Connected to server.");
});

ws.on("message", (message) => {
	const event = JSON.parse(message.toString());
	console.log(event.type);
});

Once connected, the session speaks the standard realtime event protocol: send client events like session.update, conversation.item.create, input_audio_buffer.append, and response.create; receive server events like session.created, response.output_audio.delta, and response.done.

ws.on("open", () => {
	// Configure the session.
	ws.send(
		JSON.stringify({
			type: "session.update",
			session: {
				type: "realtime",
				instructions: "You are a friendly assistant.",
				audio: {
					output: { voice: "marin" },
				},
			},
		}),
	);

	// Ask for a response.
	ws.send(
		JSON.stringify({
			type: "conversation.item.create",
			item: {
				type: "message",
				role: "user",
				content: [{ type: "input_text", text: "Say hello!" }],
			},
		}),
	);
	ws.send(JSON.stringify({ type: "response.create" }));
});

All events are JSON text frames; audio travels base64-encoded inside events (binary WebSocket frames are rejected). The model is locked at connection time — a session.update that tries to change session.model is rejected with a model_locked error event.

Browser Clients and Client Secrets

Never ship a long-lived API key to a browser. Instead, mint a short-lived ephemeral client secret from your backend with the OpenAI-compatible POST /v1/realtime/client_secrets endpoint:

curl -X POST "https://api.llmgateway.io/v1/realtime/client_secrets" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "expires_after": { "anchor": "created_at", "seconds": 120 },
    "session": {
      "type": "realtime",
      "model": "gpt-realtime"
    }
  }'

The response contains the secret (ek_...) and its expiry:

{
	"value": "ek_...",
	"expires_at": 1753500000,
	"session": {
		"type": "realtime",
		"model": "gpt-realtime"
	}
}

The browser then connects using the standard openai-insecure-api-key WebSocket subprotocol — no Authorization header needed:

const ws = new WebSocket("wss://api.llmgateway.io/v1/realtime", [
	"realtime",
	"openai-insecure-api-key." + clientSecret,
]);

Client secret behavior:

• TTL: 10–300 seconds (default 60), set via expires_after.seconds. Secrets are reusable until they expire; expiry only gates opening the connection, not the session's duration.
• Model pinning: the secret is minted for one model. The model query parameter is optional when connecting with a secret; if provided, it must match the minted model.
• Transcription pinning: optionally pin an input transcription model at mint time via session.audio.input.transcription.model. The session can then only enable that transcription model.
• Instruction pinning: optionally pin the session prompt at mint time via session.instructions — see below.
• Voice: optionally set the session's default output voice at mint time via session.audio.output.voice. Unlike instructions this is only a default; the client may still change it.
• All authentication, credit, model access, and compliance checks run at mint time and again at connection time.

Server-Authoritative Instructions

By default the session prompt is set by whoever holds the socket, so a browser client has to send session.instructions itself. If your server owns the prompt, pin it at mint time instead:

{
	"session": {
		"type": "realtime",
		"model": "gpt-realtime",
		"instructions": "You are a support agent for ACME."
	}
}

The gateway then applies the instructions itself when the session connects, so the prompt never has to travel through the browser. It is not echoed in the mint response, and it is stripped from the session.created and session.updated events the client receives.

Once pinned, the instructions are locked for the session's lifetime, the same way session.model is. A client that tries to change them gets an instructions_locked error event and the change is not applied — this covers both session.update and per-response response.create.instructions.

Instructions larger than 16,384 estimated tokens are rejected at mint time with a 400 instructions_too_large, so an oversized prompt fails server-to-server rather than mid-call.

Input Audio Transcription

Enable transcription of the user's audio with session.update. An explicit, supported transcription model is required — check the models page for available realtime transcription models and their pricing:

{
	"type": "session.update",
	"session": {
		"type": "realtime",
		"audio": {
			"input": {
				"transcription": { "model": "gpt-4o-transcribe" }
			}
		}
	}
}

Transcripts arrive as standard conversation.item.input_audio_transcription.delta and .completed events.

• The provider's implicit default (e.g. whisper-1) is not available; omitting transcription.model is rejected with transcription_model_required.
• The transcription model is pinned for the session on first use and cannot be switched afterwards; disabling transcription ("transcription": null) is always allowed.
• Both the current nested form (session.audio.input.transcription) and the legacy top-level form (session.input_audio_transcription) are accepted.
• Transcription models are billed the way their provider meters them, per token or per minute of audio, at the rates on the models page.

Transcription Sessions

For live speech-to-text without a speech model in the loop — captions, agent assist, voice notes — open a transcription session. It uses the same WebSocket endpoint and event protocol, but the session's only model is the transcription model, so you pay for transcription alone. The model is pinned at connection time, so it goes in the URL alongside intent=transcription:

wss://api.llmgateway.io/v1/realtime?intent=transcription&model=gpt-live-transcribe

The gateway applies the transcription model itself once the session is created. Configure the rest with session.update, then stream audio with input_audio_buffer.append:

{
	"type": "session.update",
	"session": {
		"type": "transcription",
		"audio": {
			"input": {
				"format": { "type": "audio/pcm", "rate": 24000 },
				"transcription": {
					"model": "gpt-live-transcribe",
					"delay": "low",
					"languages": ["en"]
				},
				"turn_detection": null
			}
		}
	}
}

Transcripts arrive as conversation.item.input_audio_transcription.delta and .completed events, exactly as in a realtime session.

• Any realtime transcription model on the models page can open a transcription session; the page also shows whether a model is billed per token or per minute of audio.
• The model is locked for the session: a session.update that names another transcription model is rejected with transcription_model_locked, and disabling transcription is rejected with transcription_model_required. Other transcription settings (delay, keywords, languages, prompt, turn_detection, noise_reduction) are passed through to the provider. Streaming transcription models segment audio continuously and reject turn_detection: send null and commit turns yourself with input_audio_buffer.commit.
• response.create is rejected with response_not_supported; a transcription session never generates.
• Browser clients mint a client secret with session.type: "transcription" and connect with the openai-insecure-api-key subprotocol as usual. The intent and model query parameters are optional with a secret, but if present they must match what the secret was minted for:

{
	"session": {
		"type": "transcription",
		"audio": {
			"input": {
				"transcription": { "model": "gpt-live-transcribe" }
			}
		}
	}
}

Session gating, limits and billing work as for realtime sessions: each completed transcription is billed before its transcript is forwarded, and a session that no longer clears the credit, spend or rate-limit checks is closed after the current turn. Streaming models transcribe audio before it is committed; audio that has already produced transcript deltas is committed by the gateway on disconnect and ahead of an input_audio_buffer.clear, so it is billed like any other turn.

Function Calling

Plain function tools work exactly as in the OpenAI Realtime API — declare them in session.update (or per response in response.create), receive response.function_call_arguments events, and return results with conversation.item.create:

{
	"type": "session.update",
	"session": {
		"type": "realtime",
		"tools": [
			{
				"type": "function",
				"name": "get_weather",
				"description": "Get the current weather for a location.",
				"parameters": {
					"type": "object",
					"properties": {
						"location": { "type": "string" }
					},
					"required": ["location"]
				}
			}
		],
		"tool_choice": "auto"
	}
}

Hosted tools (MCP servers, web search, code interpreter, etc.) are not yet available and are rejected with tool_type_not_supported.

Billing and Session Gating

Realtime sessions bill your organization's pay-as-you-go credits, or your own provider key when the project uses provider keys. Every model generation — whether triggered by your response.create or by server-side VAD — passes the gateway's authorization gates first (credits, API key status, usage limits, IAM rules), so a session cannot run past an exhausted balance. A blocked generation surfaces as a standard error event on the session instead of a response.

Default per-session safety limits:

Sessions that hit a limit are closed gracefully after in-flight responses are billed. Usage appears in your activity feed like any other request, including separate line items for input transcription.

Current Limitations

• WebSocket transport only — WebRTC and SIP are not yet supported.
• Image input is not yet supported in realtime sessions.
• Hosted tools (MCP, web search, etc.) and stored prompt references (session.prompt) are not supported; inline your instructions and function tools instead.
• Realtime requires a regular developer API key on a pay-as-you-go organization. End-user session tokens, platform keys, and DevPass/chat plan organizations are not supported yet.

Self-Hosting

Realtime is disabled by default on self-hosted deployments. Set REALTIME_INLINE=true to attach the /v1/realtime WebSocket listener (and the client-secret mint endpoint) to the gateway process — pnpm dev sets this automatically. Client secrets require Redis. See .env.example for the tunables (session caps, concurrency limits, shutdown grace period).