📚olmOCR
Toolkit for linearizing PDFs for LLM datasets/training
⭐ Very popular: 19k stars, gaining about 116 a week
View on GitHub ↗repo profile
momentum
durability
bus factor = how many people it takes to cover more than half the commits (6 months). 1 is a solo project; higher means the work is spread across a team. top-author share is the single busiest author's slice of those commits.
since we covered it
why it's a big deal
- Converts PDFs, PNGs, and JPEGs into clean Markdown while preserving reading order across multi-column layouts and stripping headers and footers.
- Handles equations, tables, and handwriting, which matters for teams building LLM training datasets from messy document scans.
- Runs at a stated cost of under $200 per million pages, making bulk document linearization practical at dataset scale.
under the hood
- Built around a 7B parameter vision language model based on Qwen2.5-VL, with the v0 dot 4 release adding synthetic data training and reinforcement learning.
- Renders documents to images, runs them through the VLM, and emits Markdown or Dolma format output.
- Written in Python and needs a recent NVIDIA GPU with at least 12GB of VRAM, with deployment via local inference, a remote vLLM server, Docker, or S3 coordinated multi-node clusters.
Radar summary, generated from the project's public sources
star history
- PR#3 8k 2025-03-05
- now 19k + 12k since first covered
curve is sampled from GitHub's star history, plus our own daily readings since we covered it; the dashed stretch is before we first covered it, the solid line since. figures at coverage are the numbers we printed then (approx.), current count is live.
understory
Better known than its recent output, coasting a little on attention.
- output, commits & releases
- clout, star velocity
output = commits & releases; clout = star velocity, both 0 to 100 monthly indices; the gap where output runs above clout is the understory. The understory →
covered in
-
Toolkit for linearizing PDFs for LLM datasets/training
similar projects
compare these →- 🖥️ Open WebUI
7.7× the stars
User-friendly AI Interface (Supports Ollama, OpenAI API,...)
149k ACTIVE - 🧪 autoresearch
4.9× the stars
AI agents running research on single-GPU nanochat training automatically
94k ACTIVE - 🦙 LLaMA-Factory
3.8× the stars
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
74k ACTIVE
comments
Sign in with GitHub to add your blip on olmOCR.