🕷️Scrapling
D4Vinci/Scrapling · homepage ↗
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
⭐ Hugely popular: 74k stars, gaining about 1k a week
View on GitHub ↗repo profile
momentum
durability
bus factor = how many people it takes to cover more than half the commits (6 months). 1 is a solo project; higher means the work is spread across a team. top-author share is the single busiest author's slice of those commits.
since we covered it
why it's a big deal
- Adaptive parsing via auto_save=True means your CSS selectors survive site redesigns automatically, the framework uses similarity algorithms to find relocated elements without you rewriting anything.
- The built-in MCP server lets AI tools like Claude or Cursor directly use Scrapling to extract targeted content before passing it to the AI, reducing token usage and speeding up operations. GitHub It’s a natural fit for agentic pipelines that need reliable web data.
- 92% test coverage and full type hints coverage, and it’s been used daily by hundreds of web scrapers over the past year GitHub, this is production-grade, not a weekend project.
under the hood
- Three fetcher classes: Fetcher for fast HTTP with browser TLS fingerprint impersonation, StealthyFetcher for advanced anti-bot bypass, and DynamicFetcher for full Playwright-based browser automation.
- The v0.4 Spider framework adds a Scrapy-like API with async parse callbacks, multi-session routing, checkpoint-based pause/resume, and streaming mode for real-time stats. GitHub.
- Full Docker image with all browsers pre-installed, an interactive IPython scraping shell, and a scrapling extract CLI for zero-code URL scraping from the terminal.
our take from PR#29, 2026-03-11
star history
- PR#29 25k 2026-03-11
- now 74k + 49k since first covered
curve is sampled from GitHub's star history, plus our own daily readings since we covered it; the dashed stretch is before we first covered it, the solid line since. figures at coverage are the numbers we printed then (approx.), current count is live.
understory
Better known than its recent output, coasting a little on attention.
- output, commits & releases
- clout, star velocity
output = commits & releases; clout = star velocity, both 0 to 100 monthly indices; the gap where output runs above clout is the understory. The understory →
covered in
-
The Web Scraper That Doesn’t Break When Sites Change
similar projects
compare these →- 🔥 Firecrawl
TypeScript · 2.3× the stars · AGPL-3.0 licensed
The context API to search, scrape, and interact with the web at scale. 🔥
168k ACTIVE - 🕸️ Scrapegraph-ai
leaner, 30k stars
Python scraper based on AI
30k ACTIVE - 🎯 Skyvern
leaner, 23k stars · AGPL-3.0 licensed
Automate browser based workflows with AI
23k ACTIVE
replaces
alternatives to Apify →- Apify
market leader
closed source - Bright Data closed source
- Diffbot closed source
comments
Sign in with GitHub to add your blip on Scrapling.