🐣NanoVLM
The simplest, fastest repository for training/finetuning small-sized VLMs.
⭐ Popular: 5k stars, gaining about 6 a week
View on GitHub ↗repo profile
momentum
durability
since we covered it
why it's a big deal
- Gives newcomers a complete, readable path to training a vision-language model, positioned as the nanoGPT of VLMs rather than a state-of-the-art system.
- Lowers the hardware barrier, with a 222M-parameter model trainable in roughly 6 hours on a single H100 and inference fitting in about 4GB of VRAM at batch size 1.
- Serves researchers, students, and hobbyists who want to understand and modify every part of a VLM instead of treating one as a black box.
under the hood
- Pure PyTorch implementation in Python, with no Trainer, accelerate, or deepspeed, spread across roughly 750 lines of core logic.
- Combines a SigLIP-B/16-224-85M vision encoder with the SmolLM2-135M language decoder, joined by a small modality projection module.
- Trains on about 1.7M samples from the cauldron and supports Hub loading and saving, lmms-eval evaluation, DDP multi-GPU training, and image splitting for higher resolutions.
Radar summary, generated from the project's public sources
star history
- PR#8 2k 2025-05-14
- now 5k + 3k since first covered
curve is sampled from GitHub's star history, plus our own daily readings since we covered it; the dashed stretch is before we first covered it, the solid line since. figures at coverage are the numbers we printed then (approx.), current count is live.
understory
Better known than its recent output, coasting a little on attention.
- output, commits & releases
- clout, star velocity
output = commits & releases; clout = star velocity, both 0 to 100 monthly indices; the gap where output runs above clout is the understory. The understory →
covered in
-
Minimalist Vision-Language Model in Pure PyTorch
similar projects
compare these →- 🖥️ Open WebUI
30× the stars
User-friendly AI Interface (Supports Ollama, OpenAI API,...)
149k ACTIVE - 🧪 autoresearch
19× the stars
AI agents running research on single-GPU nanochat training automatically
94k ACTIVE - 🦙 LLaMA-Factory
15× the stars
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
74k ACTIVE
comments
Sign in with GitHub to add your blip on NanoVLM.