Source repository · Star if this helps

LiteLLM: Multi-Model Gateway, Quotas, and Fallback

Virtual keys, tenant quotas, and fallback, with existing Java clients on one gateway

Written from the LiteLLM docs and the Admin UI of the official main-stable image on 2026-11-12 (mock upstream, PostgreSQL 16, no vendor keys). Sidebar shows v1.169.0; the success header from the same run is x-litellm-version: 1.109.0 · Online edition

2026-11-12

Part 1 Why the platform needs a gateway

Business services should not hold model keys Run the LiteLLM Proxy

Part 2 Keys, quotas, and tenants

Virtual keys Team budgets and rate limits Where tenant isolation stops at the gateway

Part 3 Routing and resilience

Aliases, deployments, and the model name the business sees Fallback, retries, and load

Part 4 Wiring it into the platform you already have

Point AgentScope and Spring AI at the same entry Send call traces to Langfuse Production boundaries