🗂️pg lake
Snowflake-Labs/pg_lake · homepage ↗
pg_lake: Postgres with Iceberg and data lake access
⭐ Emerging: 2k stars, gaining about 10 a week
View on GitHub ↗repo profile
momentum
durability
bus factor = how many people it takes to cover more than half the commits (6 months). 1 is a solo project; higher means the work is spread across a team. top-author share is the single busiest author's slice of those commits.
since we covered it
why it's a big deal
- Lets Postgres users create, query, and modify Apache Iceberg tables and read data lake files in Parquet, CSV, JSON, and Iceberg formats directly through SQL, without moving to a separate warehouse.
- Combines heap tables, Iceberg tables, and external object-storage files in a single query, so teams can keep transactional Postgres and a lakehouse behind one interface.
- Originated as Crunchy Data Warehouse and was open-sourced by Snowflake in November 2025 under Apache 2.0, giving existing Crunchy Bridge for Analytics functionality a public codebase.
under the hood
- Splits into two components, Postgres with pg_lake extensions for connections, planning, and transactions, and a standalone multi-threaded pgduck_server that speaks the Postgres wire protocol and runs queries on DuckDB.
- Ships as focused extensions including pg_lake_iceberg for the Iceberg v2 spec, pg_lake_table as a foreign data wrapper over object storage, and pg_lake_copy for COPY to and from data lakes.
- Written mainly in C and Python, depends on Apache Avro and DuckDB with build-time patches, and handles S3 credentials through DuckDB's secrets manager including S3-compatible stores like MinIO.
Radar summary, generated from the project's public sources
star history
- PR#22 1k 2025-11-26
- now 2k + 415 since first covered
curve is sampled from GitHub's star history, plus our own daily readings since we covered it; the dashed stretch is before we first covered it, the solid line since. figures at coverage are the numbers we printed then (approx.), current count is live.
understory
Deep in the understory, shipping hard, still flying under the radar.
- output, commits & releases
- clout, star velocity
output = commits & releases; clout = star velocity, both 0 to 100 monthly indices; the gap where output runs above clout is the understory. The understory →
covered in
-
Postgres as a lakehouse engine
similar projects
compare these →- 🛣️ Pathway
39× the stars
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
62k ACTIVE - 🛣️ LiteLLM
35× the stars
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
56k ACTIVE - 🗒️ Paperless-ngx
27× the stars · GPL-3.0 licensed
A community-supported supercharged document management system: scan, index and archive all your documents
44k ACTIVE
comments
Sign in with GitHub to add your blip on pg lake.