Problem
Downstream teams worked from raw Postgres change-data dumps rather than a consistent, typed source.
Approach
Land raw events in bronze, clean and type them into 40 silver tables refreshed every 15 minutes, then build curated data marts for analytics, AI agents and machine learning.
Measured outcome
Worst-case freshness: ~32 → ~18 minutes; zero row loss
Tech stack
- Lake/Storage
- Apache IcebergR2 Data Catalog
- Ingest & CDC
- Postgres CDC
- Transform
- DuckDBCloudflare WorkersCloudflare Containers