⚡FlashInfer
flashinfer-ai/flashinfer · homepage ↗
FlashInfer: Kernel Library for LLM Serving
⭐ Popular: 6k stars, gaining about 48 a week
View on GitHub ↗repo profile
momentum
durability
why it's a big deal
- A performance layer for LLM serving: hand-tuned GPU kernels that cut inference cost and latency without touching model code.
- Already infrastructure rather than a demo, pulled over a million times a week inside production inference stacks.
- A clear example of value in AI moving down to the kernel and serving layer, not just the models running on top.
under the hood
- CUDA and JIT kernel library for LLM serving, focused on attention and other latency-critical operations.
- Built to plug into PyTorch and NVIDIA GPUs, and used as a building block inside larger inference engines.
- Distributed on PyPI and pulled roughly 1.1M times a week.
our take from PR#38, 2026-07-15
star history
- PR#38 6k 2026-07-15
- now 6k + 86 since first covered
curve is sampled from GitHub's star history, plus our own daily readings since we covered it; the dashed stretch is before we first covered it, the solid line since. figures at coverage are the numbers we printed then (approx.), current count is live.
understory
Deep in the understory, shipping hard, still flying under the radar.
- output, commits & releases
- clout, star velocity
output = commits & releases; clout = star velocity, both 0 to 100 monthly indices; the gap where output runs above clout is the understory. The understory →
covered in
-
the neutral kernel layer under your inference stack
similar projects
compare these →- 🦙 LLaMA-Factory
12× the stars
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
74k ACTIVE - ☁️ skypilot
The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.
10k ACTIVE - 🖥️ Open WebUI
24× the stars
User-friendly AI Interface (Supports Ollama, OpenAI API,...)
147k ACTIVE
comments
Sign in with GitHub to add your blip on FlashInfer.