🛣️Pathway
pathwaycom/pathway · homepage ↗
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
⭐ Hugely popular: 62k stars, gaining about 470 a week
View on GitHub ↗repo profile
momentum
durability
bus factor = how many people it takes to cover more than half the commits (6 months). 1 is a solo project; higher means the work is spread across a team. top-author share is the single busiest author's slice of those commits.
since we covered it
why it's a big deal
- Unified Batch & Streaming: Write once, run anywhere - Pathway’s architecture allows the same codebase to handle both batch and streaming data seamlessly.
- LLM & RAG Integration: Built-in support for LLMs and RAG pipelines, including vector indexing and connectors to services like LangChain and LlamaIndex.
- Robust Connectors: Out-of-the-box connectors for Kafka, PostgreSQL, SharePoint, Google Drive, and more, with support for over 300 data sources via Airbyte.
under the hood
- Python API: Offers an easy-to-use Python interface, compatible with popular ML libraries and tools.
- Incremental Computation: Utilizes Differential Dataflow for efficient, real-time updates and processing.
- Deployment Flexibility: Easily deployable via Docker and Kubernetes, with built-in observability and monitoring tools.
our take from PR#9, 2025-05-28
star history
- PR#9 25k 2025-05-28
- now 62k + 37k since first covered
curve is sampled from GitHub's star history, plus our own daily readings since we covered it; the dashed stretch is before we first covered it, the solid line since. figures at coverage are the numbers we printed then (approx.), current count is live.
understory
Quietly building: more output than attention, for now.
- output, commits & releases
- clout, star velocity
output = commits & releases; clout = star velocity, both 0 to 100 monthly indices; the gap where output runs above clout is the understory. The understory →
covered in
-
Python Framework for Real-Time Data & AI Pipelines
similar projects
compare these →- 🥥 CocoIndex
Rust · leaner, 11k stars
Incremental engine for long horizon agents 🌟 Star if you like it!
11k ACTIVE - 🛣️ LiteLLM
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
56k ACTIVE - 🧩 Pathway LLM App
Jupyter Notebook
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.
59k ACTIVE
comments
Sign in with GitHub to add your blip on Pathway.