Using TanStack AI

TanStack AI ships a first-party LLM Gateway adapter: @tanstack/ai-llmgateway, maintained in the TanStack AI repository alongside the OpenAI and Anthropic adapters. One adapter and one API key reach every model in the catalog.

Install

pnpm add @tanstack/ai @tanstack/ai-react @tanstack/ai-llmgateway

@tanstack/ai-react is the React client; TanStack AI also ships Vue, Svelte, Angular, and Preact packages that work with the same adapter.

Set your API key (create one from the dashboard):

export LLM_GATEWAY_API_KEY=llmgtwy_your_key_here

Stream chat from a server route

llmGatewayText(model) creates the adapter and reads the key from LLM_GATEWAY_API_KEY:

// app/api/chat/route.ts
import { chat, toServerSentEventsResponse } from "@tanstack/ai";
import { llmGatewayText } from "@tanstack/ai-llmgateway";

export async function POST(request: Request) {
	const { messages } = await request.json();

	const stream = chat({
		adapter: llmGatewayText("gpt-5.6-terra"),
		messages,
	});

	return toServerSentEventsResponse(stream);
}

Switching models is a one-line change — the same adapter serves every model:

adapter: llmGatewayText("claude-sonnet-5"),

Model ID formats

LLM Gateway supports two model ID formats:

Canonical model IDs (gpt-5.6-terra) — smart routing picks the best provider based on uptime, throughput, price, and latency
Provider-prefixed IDs (moonshot/kimi-k3) — routes to a specific provider with automatic failover if uptime drops below 90%

A curated set of flagship models carries typed metadata (LLMGATEWAY_CHAT_MODELS) with editor autocomplete for input modalities and options; any other ID from the models page still works. See the routing documentation for details.

Connect the React client

useChat consumes the AG-UI event stream from the route above — no client-side API key, no per-provider wiring:

// components/chat.tsx
"use client";

import { fetchServerSentEvents, useChat } from "@tanstack/ai-react";
import { useState } from "react";

export function Chat() {
	const [input, setInput] = useState("");
	const { messages, sendMessage, isLoading } = useChat({
		connection: fetchServerSentEvents("/api/chat"),
	});

	return (
		<div>
			{messages.map((message) => (
				<div key={message.id}>
					<strong>{message.role === "assistant" ? "Assistant" : "You"}</strong>
					{message.parts.map((part, index) =>
						part.type === "text" ? <p key={index}>{part.content}</p> : null,
					)}
				</div>
			))}
			<form
				onSubmit={(event) => {
					event.preventDefault();
					if (!input.trim() || isLoading) {
						return;
					}
					sendMessage(input);
					setInput("");
				}}
			>
				<input
					value={input}
					onChange={(event) => setInput(event.target.value)}
					placeholder="Say something..."
				/>
			</form>
		</div>
	);
}

Tool calling

Define tools with toolDefinition and attach a server handler — TanStack AI runs the tool loop for you:

import { chat, toServerSentEventsResponse, toolDefinition } from "@tanstack/ai";
import { llmGatewayText } from "@tanstack/ai-llmgateway";
import { z } from "zod";

const getWeather = toolDefinition({
	name: "get_weather",
	description: "Get the current weather for a location",
	inputSchema: z.object({
		location: z.string(),
	}),
}).server(async ({ location }) => {
	return { temperature: 72, condition: "sunny" };
});

export async function POST(request: Request) {
	const { messages } = await request.json();

	const stream = chat({
		adapter: llmGatewayText("gpt-5.6-terra"),
		messages,
		tools: [getWeather],
	});

	return toServerSentEventsResponse(stream);
}

Reasoning models

Reasoning models stream their thinking as reasoning_content deltas, which the adapter surfaces as AG-UI REASONING_* events — they arrive in useChat as thinking parts. Control the depth with reasoning_effort in modelOptions:

const stream = chat({
	adapter: llmGatewayText("kimi-k3"),
	messages,
	modelOptions: {
		temperature: 0.7,
		reasoning_effort: "high",
	},
});

reasoning_effort accepts the extended scale none / minimal / low / medium / high / xhigh / max; which tiers a model honors depends on the model and the provider it routes to. Parameters a routed provider doesn't support are stripped server-side, so modelOptions stay portable across models. See reasoning support.

Summarization

The adapter also covers TanStack AI's summarize surface:

import { summarize } from "@tanstack/ai";
import { llmGatewaySummarize } from "@tanstack/ai-llmgateway";

const result = await summarize({
	adapter: llmGatewaySummarize("gpt-5.4-mini"),
	text: "Long article text...",
	stream: false,
});

Self-hosted deployments

createLLMGatewayText takes the key explicitly plus a baseURL for self-hosted gateways:

import { createLLMGatewayText } from "@tanstack/ai-llmgateway";

const adapter = createLLMGatewayText(
	"gpt-5.6-terra",
	process.env.LLM_GATEWAY_API_KEY!,
	{
		baseURL: "https://gateway.internal.example.com/v1",
	},
);

Every request made through TanStack AI shows up in your Activity and Usage & Metrics dashboards like any other gateway request — with per-request cost, tokens, and latency.