Skip to content
repository radar
Infra, Data & Ops

🛣️LiteLLM

ACTIVE STEADY

BerriAI/litellm · homepage ↗

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

⭐ Hugely popular: 56k stars, gaining about 582 a week

View on GitHub ↗

repo profile

vintage 2023 3 years old
delivery product
language Python
license Other

momentum

total stars 56k
stars added last week +582
HN peak 938 pts 4mo ago
weekly downloads 196M /wk · PyPI
used by 24k repos & packages
commits / week 445 steady · -7% vs prior mo
issues closed 80% ~3d to close, median
contributors 2k (+31 in 15d)
release cadence weekly
last activity today

durability

backing VC-backed LiteLLM / BerriAI (YC W23)
security score 5.8/10 OpenSSF Scorecard · 2026-08-13
bus factor 5 distributed
top-author share 16% 6 mo

bus factor = how many people it takes to cover more than half the commits (6 months). 1 is a solo project; higher means the work is spread across a team. top-author share is the single busiest author's slice of those commits.

since we covered it

monthly average + 4k/mo (+8%/mo) · + 10k total since PR#34

why it's a big deal

  • With Portkey heading into Palo Alto's Prisma AIRS, LiteLLM is the most mature OSS analog that stays self-hostable and provider-agnostic.
  • It treats the AI gateway as a primitive, not a product - virtual keys, per-team spend ledgers, and provider fallback shipped as a library you run yourself.
  • Battle-tested at over 10B tokens and shipping roughly 394 pull requests a month, it is the gateway layer small teams and enterprise alike keep reaching for.

under the hood

  • Dual-mode: drop-in Python library (litellm.completion(...)) or standalone FastAPI proxy server with admin UI.
  • Per-provider adapter pattern translates OpenAI-format requests and streams to native provider SDKs, with first-class guardrails, retries, and load balancing.
  • Multi-tenant from the start - virtual API keys, team and user spend tracking, per-key model allow-lists.

our take from PR#34, 2026-05-20

star history

PR#34 · 46k56k now Jul 2023Aug 2026
  1. PR#34 46k 2026-05-20
  2. now 56k + 10k since first covered

curve is sampled from GitHub's star history, plus our own daily readings since we covered it; the dashed stretch is before we first covered it, the solid line since. figures at coverage are the numbers we printed then (approx.), current count is live.

understory

Quietly building: more output than attention, for now.

+17 understory score output 100 · clout 83
Aug 2025 Jul 2026
  • output, commits & releases
  • clout, star velocity

output = commits & releases; clout = star velocity, both 0 to 100 monthly indices; the gap where output runs above clout is the understory. The understory →

covered in

  • PR#34 2026-05-20 above the radar

    The OSS AI gateway that stays independent post-Portkey

similar projects

compare these →
  • 🔀 OmniRoute

    TypeScript

    Never stop coding. Free MIT AI gateway: one endpoint, 339 providers (90+ free), 1200+ models, Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A, Desktop/PWA. Built by 450+ contributors

    49k ACTIVE
  • 🧩 TensorZero

    Rust · leaner, 12k stars

    TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

    12k OFF THE RADAR
  • 🗜️ headroom

    Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

    66k ACTIVE

comments

Sign in with GitHub to add your blip on LiteLLM.

loading comments…