Apache Iceberg Fundamentals

Why Iceberg?

Iceberg addresses long-standing issues of big data: slow queries, complex schema changes, and the tight coupling of storage with computing systems. You'll learn why companies like Netflix, Stripe, and Apple have chosen Iceberg for their platforms and how to apply these approaches in your own setup.

What you will do:

1. Build a local Lakehouse lab based on Iceberg using Docker Compose, Spark, REST catalog, and MinIO.
2. Create your first Iceberg table using a fun dataset (like one with Pokémon), define the schema, write data through PySpark, and explore how Iceberg manages metadata.
3. Master schema evolution: adding, renaming, and changing column types, as well as advanced partitioning techniques.
4. Learn to perform point-in-time operations (such as deleting rows) and use the "time travel" feature to analyze past versions of data.
5. Dive into Iceberg's architecture: parquet files, manifests, snapshots, and catalogs.
6. Use the MinIO UI to see how data and metadata are physically stored.
7. Run analytical SQL queries on Iceberg tables through PySpark, using familiar operations like join, group by, and filter.