Q1. Explain the Medallion Architecture (Bronze, Silver, Gold) in a Data Lake.
The Medallion Architecture is a multi-layer data quality framework. The Bronze layer is our raw ingestion zone -- data lands here exactly as it arrives from source systems, completely unmodified. The Silver layer is where we cleanse, validate, deduplicate, and join data -- it is the single source of truth for data engineers. The Gold layer contains business-domain aggregations and pre-built facts and dimensions ready for direct consumption by BI tools and data scientists. This progression ensures we can always reprocess from raw if business rules change, without re-ingesting from source systems.
Q2. What is Delta Lake and what problem does it solve?
Delta Lake is an open-source storage format built on Apache Parquet that adds a transaction log to enable ACID guarantees on your data lake. Without Delta, a Parquet-based lake suffers from the small file problem, no schema enforcement, no upserts, and no rollback capability. Delta solves all of this: you can run UPDATE, DELETE, and MERGE (upsert) operations, enforce schema evolution rules, time-travel to any historical snapshot using versioned transaction logs, and automatically compact small files using OPTIMIZE. This transforms a simple storage bucket into a production-grade lakehouse.