Apache Iceberg: Complete Hands-On Masterclass

Name: Apache Iceberg: Complete Hands-On Masterclass
Rating: 5.0 (6 reviews)

Master Apache Iceberg with hands-on labs using PySpark, Databricks, and Google Colab—no setup or data needed

Role Play

Created byDE & AI Engineering Simplified

Last updated 7/2025

English

What you'll learn

Learn why Apache Iceberg is redefining modern data lakes and how it overcomes limitations of Hive, Delta Lake, and Hudi.
Set up Iceberg in Databricks & Google Colab—no local install or cloud budget needed—and start building hands-on from your browser.
Perform DDL and DML operations (insert, update, delete) on Iceberg tables and explore internal metadata structures such as snapshots, manifests, and partitions.
Master Iceberg’s time travel and metadata tables to build scalable, version-controlled, cost-efficient data pipelines.
Understand Iceberg’s architecture under the hood—how it handles schema, partition evolution, and decouples compute from storage.
Explore real-world debugging, rollback, and auditing use cases using Iceberg’s powerful snapshot and metadata tracking system.

Course content

8 sections • 30 lectures • 4h 14m total length

Why To Learn Apache Iceberg6:07
Course Roadmap: What You’ll Learn, Step by Step3:10
Course Technical Pre-requisite2:01
Downloadable Resources: Diagrams, Code & Data Files2:33

Introduction - File Format & Table Format With Apache Spark2:17
Databricks Community Edition - Self Registration1:17
Databricks Web Interface Walkthrough3:50
Databricks Freemium Edition - Support For Iceberg Table & Limitations3:48
Create Compute & Notebook in Databricks10:15
Apache Spark With Parquet File Format11:14
This chapter demonstrates a PySpark program that creates dummy inventory data, writes it to parquet and CSV, partitions by category, and contrasts file format with the Iceberg table approach.
Quiz - PySpark Data Handling Fundamentals
PySpark With Iceberg File Format26:30
Quiz - PySpark + Apache Iceberg: Fundamentals

PySpark On Google Colab Notebook1:41
Learn to run PySpark with Iceberg on Google Colab by setting up Spark and Hadoop, checking PySpark config, and creating and validating the inventory table and metadata files.
Free Google Colabe Notebook Setup To Run PySpark6:31
Apache Iceberg With PySpark On Colabe Notebook14:57
Choosing the Right Platform: Databricks CE, Colab, or Databricks Freemium for Apache Iceberg PoC

OLAP & Role Of File & Table Formats7:07
Data Lake (To Be modified)2:09
Hive Table Format7:52
Create two external tables over a csv file and a parquet file in Databricks using a Hive table format with header and inferred schema, enabling seamless queries via the catalog.
Data Lake2:49
Why Iceberg Table Format3:38
Discover how lakehouse combines storage and compute, and how the iceberg table format enables ac transactions, time travel, and partition evolution through explicit configuration.
Convince Me: Why File Format & Table Format Matter in Spark-Based Datalake?

Create & Insert Operation With Apache Iceberg Table16:32
Insert Operation (DML) With Apache Icerberg10:24
Delete Data From Iceberg Table With Spark 3.222:39
Delete Data From Iceberg Table With Spark 3.5 (Non Partitioned Data)17:11
Demonstrates deleting records from an Apache Iceberg table without partitioning using Spark 3.5 in a Google Colab notebook, and analyzes resulting metadata and data file changes.
Iceberg Built-in Metadata Tables14:21
Explore iceberg built-in metadata tables—history, metadata log entries, snapshots, and manifests—and see how inserts, deletes, and updates reveal data evolution, enabling time travel and strong observability.
Time-Travel Feature With Iceberg4:20
Master the time travel feature of Apache Iceberg tables by using version as of or timestamp as of to view and roll back data via snapshot IDs and metadata table.

Requirements

Basic understanding of Python programming is helpful.
Some familiarity with PySpark and big data file formats like Parquet is recommended.
Familiarity with file formats like Parquet or ORC is useful but not mandatory.
No need for cloud subscriptions or local installations — Databricks Community Edition and Google Colab are used (both are free).

Description

This course offers a practical, hands-on introduction to Apache Iceberg, the modern open table format designed for today’s large-scale data lakes and lakehouses. Whether you’re a data engineer, developer, or architect, this course will help you understand and apply Iceberg concepts through real-world exercises—without the need for any infrastructure setup.

You’ll learn to create, query, and manage Iceberg tables using PySpark in both Databricks Community Edition and Google Colab—two free platforms accessible from your browser. We cover everything from understanding table formats, DDL and DML operations, partition evolution, schema evolution, metadata tables, and Iceberg’s powerful time travel capability.

All code and sample data are provided chapter by chapter. You’ll generate data on the fly, inspect table structures, and compare metadata files using VS Code and online JSON viewers. No local installation, no external datasets—just clear, interactive learning.

What You’ll Learn

Key differences between file formats and table formats in big data
How to create and manage Apache Iceberg tables using PySpark
Comparing Hive tables and Iceberg with practical demos
Running Iceberg on Databricks and Google Colab (setup included)
Performing DDL and DML operations (insert, update, delete)
Using Iceberg’s built-in metadata tables to inspect file-level and snapshot info
Implementing time travel to query historical data versions
Understanding how Iceberg handles schema evolution and partition changes
Comparing Iceberg with Delta Lake and Hudi in practical scenarios

By the end of the course, you’ll have a strong working knowledge of Apache Iceberg and be ready to use it in real-world data projects with confidence.

Who this course is for:

This course is designed for data engineers, data architects, and developers who work with large-scale data pipelines and are looking to modernize their data lake or lakehouse architecture. If you’re currently using Hive, Delta Lake, or Hudi and want to explore a more flexible, scalable, and engine-agnostic table format, this course is for you.
It’s also ideal for those who want to gain hands-on experience with Apache Iceberg using free tools like Databricks Community Edition and Google Colab — without needing complex infrastructure or cloud setup.
Whether you’re building batch or streaming pipelines, working on schema evolution, or just want to understand Iceberg’s time travel and metadata capabilities, this course will help you build practical skills with real-world applications.
No prior experience with Iceberg is required — just a willingness to learn and some basic familiarity with Python or PySpark.

Apache Iceberg: Complete Hands-On Masterclass

What you'll learn

Explore related topics

Course content

Introduction To Apache Iceberg4 lectures • 14min

How PySpark Works With File Format & Table Format - Using Databricks Notebook7 lectures • 59min

Apache Iceberg Hands On Using Google Colab4 lectures • 23min

Apache Iceberg Architecture - Deep Dive6 lectures • 24min

Apache Iceberg Hands on Lab & Deep Dive6 lectures • 1hr 25min

Thank You1 lecture • 1min

Bonus Section - Advanced Topic2 lectures • 34min

Bonus Section - Iceberg System Procedures2 lectures • 14min

Requirements

Description

Who this course is for: