Introduction to LLM Gateway
LLM Gateway is an open-source API gateway for Large Language Models. Route requests to multiple providers, manage API keys, track usage, and optimize costs.
LLM Gateway is an open-source API gateway that sits between your applications and LLM providers like OpenAI, Anthropic, Google AI Studio, and more. It provides a unified, OpenAI-compatible API interface with built-in cost tracking, caching, and intelligent routing.
How it works
Point your existing SDK at https://api.llmgateway.io/v1 (or your self-hosted instance) and authenticate with an LLM Gateway API key — no code rewrites. The gateway routes each request to the right provider, tracks tokens and cost per model, provider, project, and API key, and fails over to a healthy provider when one errors. Pay per-token with prepaid credits at provider list rates, or bring your own provider keys (BYOK) for free.
Why use a gateway?
- One integration — switch models or providers by changing a model string, not your code.
- Cost visibility — usage analytics and cost breakdowns across every provider in one dashboard.
- Reliability — automatic provider failover and response caching in front of every request.
- No lock-in — open source (AGPLv3), self-hostable, and OpenAI-compatible end to end.
Features
Routing
Intelligently route requests to the best available models and providers.
Caching
Reduce costs and latency by caching identical requests.
Image Generation
Generate images using AI models through the OpenAI-compatible API.
Video Generation
Generate videos using the OpenAI-compatible async video API.
Web Search
Enable real-time web search capabilities for up-to-date information.
Reasoning
Use reasoning-capable models that show their step-by-step thought process.
AI Tooling
LLM Gateway is built to work seamlessly with AI agents and development tools.
llms.txt
Machine-readable index of all documentation pages for LLM consumption.
llms-full.txt
Complete documentation content in a single file for full-context LLM ingestion.
MCP Server
Use LLM Gateway as an MCP server for Claude Code, Cursor, and other MCP-compatible clients.
Agent Skills
Packaged instructions and guidelines for AI coding agents to generate higher-quality code.
Templates & Agents
Pre-built templates and agent configurations to get started quickly.
Next Steps
- Quickstart — Get up and running in minutes
- Overview — Learn more about what LLM Gateway offers
- Self-Host — Deploy on your own infrastructure
How is this guide?
Last updated on