Self Host LLM Gateway

Deploy LLM Gateway on your own infrastructure with Docker, Docker Compose, or Kubernetes on AWS, GCP, and Azure.

LLM Gateway is a self-hostable platform that provides a unified API gateway for multiple LLM providers. Run it on your own infrastructure to keep full control over your data and avoid platform fees.

Pick the deployment path that matches where you're running it.

Single host

Docker

Run every service in one container. The fastest way to a working instance.

Docker Compose

Run each service in its own container for more control over scaling and config.

Cloud

Kubernetes (Helm)

Deploy the official Helm chart to any cluster — EKS, GKE, AKS, or your own.

AWS

EKS, RDS for Postgres, ElastiCache, and Secrets Manager.

Google Cloud

GKE, Cloud SQL, Memorystore, and Secret Manager.

Azure

AKS, Azure Database for PostgreSQL, Azure Cache for Redis, and Key Vault.

Which option should I choose?

Trying it out or running a single low-traffic instance? Start with Docker or Docker Compose on one machine.
Running in production? Deploy to Kubernetes with our Helm chart, and use a managed Postgres and Redis from your cloud.
On a specific cloud? Follow the AWS, Google Cloud, or Azure guide for the exact managed services to provision.

What you'll need

Every deployment is built from the same pieces:

Stateless services — the gateway, API, UI, and a background worker. Scale these freely; they hold no data between requests.
PostgreSQL — the source of truth for users, projects, keys, and usage records.
Redis — response caching and the queue that feeds the worker.
Provider API keys — the OpenAI, Anthropic, Google, and other credentials the gateway uses, injected as secrets.

In production, run PostgreSQL and Redis as managed services from your cloud provider so backups, failover, and patching are handled for you.