Apache Airflow | A Real-Time & Hands-On Course on Airflow

Name: Apache Airflow | A Real-Time & Hands-On Course on Airflow
Rating: 4.6 (3045 reviews)

Learn A to Z of Apache Airflow from Basic to ADVANCE level, deploy workflows & data pipelines using Airflow with Docker

Bestseller

Created byA to Z Mentors

Last updated 5/2025

English

What you'll learn

Learn Full In & out of Apache Airflow with proper HANDS-ON examples from scratch.
Start with the implementation of Airflow core nomenclature - DAG, Operators, Tasks, Executors, Cfg file, UI views etc.
ADVANCE Airflow concepts, the explanation to which is not very clear even in Airflow's Official Documentation.
XComs, Hooks, Pools, SubDAGs, Variables, Connections, Plugins, Adhoc queries, Branching, Sensors and many more....
Exclusive features - Data Profiling, Charts, Trigger rules, airflowignore file, Zombies, Undeads, LatestOnly operator.
Create data pipeline of Real-Time case studies using Apache Airflow.
Learn Best practices / Do's & Don'ts to follow in Real-Time Airflow Projects.
Codes and Data-sets are available in resources tab.

Course content

18 sections • 63 lectures • 6h 2m total length

Introduction to Apache Airflow3:41
Welcome to this first lecture of Apache Airflow course. From this lecture we will start our journey to learn Apache Airflow. And in this lesson you will learn about the basics of Apache Airflow.
Conventional Scheduling Approaches5:31
Here we will see what were the other conventional scheduling approaches that people used before the commence of Airflow.
Why move to Airflow5:34
After seeing the drawbacks of conventional schedulers like Crons, in this video we will learn why should one move from the conventional scheduling approaches to the all new Apache Airflow scheduler.
Basic Terminologies in Airflow5:32
In this Airflow lecture, the basic terminologies like DAG, Workflow, Tasks, and few other commonly used terms in Airflow are explained.

First look of Airflow UI7:41
In this, we will have a first look of Apache Airflow UI and will get to know about its basic components.
Running default DAG in UI6:45
In this lecture we will run the default DAG in UI and will understand how a data pipeline is executed by Airflow.
Views in UI (Tree, Graph, Gantt, etc)8:09
After getting familiar with Airflow UI, in this lecture we will learn about different views that Airflow provides like Graph view, Tree view, Gantt view and more.
Understanding DAG definition file11:25
Here we will understand the code of default DAG and will learn how the workflows are written in Airflow in the form of DAGs.
DAG file execution2:59
So after knowing the underlying code of tutorial DAG, in this lecture we will run that DAG in UI and will understand its execution with context to tasks written in it.
Quiz 1

Project Requirements3:39
In this first Airflow project, we will understand what are the project requirements.
Writing Project Compose File8:05
In this lecture we will write the compose file for our project and will launch the Airflow environment that is tailor made as per our project requirements.
Writing Project's DAG File13:42
In this lecture we will write the DAG definition file for the store sales project.
Creating Connections in UI4:25
In this lecture we will learn how to create connections from Airflow UI.
Writing DAG continued10:54
After creating connections, in this lecture, we will continue with writing the workflow for our Store Sales project.
Running project DAG in UI3:24
Now that our DAG file is ready, in this video we will run that DAG in Airflow UI and will see the results.
'with' - Context Manager2:19
In this lecture we will implement using a 'with' context manager in the DAG file.

What are Executors2:09
In this lecture what are executors and what are there types is explained.
Sequential Executor10:37
In this lecture we will learn about the default executor of Airflow, that is, Sequential Executor
Local Executor6:22
After learning about Sequential Executor, in this video we will learn about an important executor that helps you in achieving task parallelism i.e., Local Executor.
Celery Executor7:39
Here we will discuss about Celery Executors in Airflow.

XComs in Airflow3:35
In this lesson we will learn how we can transfer messages from one task to another using XComs.
Implementing XComs - Part 14:09
Here we will use the knowledge gained from above theory lecture and will use the Airflow's XComs feature in DAG.
Implementing XComs - Part 24:42
In this lecture we will continue implementing XComs in DAG.
Variables in Airflow5:24
In this lecture we will see how we can create Variables from Airflow UI.
Dummy Operator4:02
In this class we will see what are the Dummy Operators in Airflow.
Quiz 2

Requirements

A very basic knowledge of Python will be an add-on.
Rest everything on Airflow is covered in this course with line to line explanations.

Description

"Apache Airflow is a platform created by community to programmatically author, schedule and monitor workflows."

Airflow is going to change the way of scheduling data pipelines and that is why it has become the Top-level project of Apache. In fact, it has already been adopted by mass companies.

What's included in the course ?

Complete Apache Airflow concepts explained from Scratch to ADVANCE with Real-Time implementation.
Each and every Airflow concept is explained with HANDS-ON examples.
Includes each and every, even thin detail of Airflow.
XComs, Hooks, Pools, SubDAGs, Variables, Connections, Plugins, Adhoc queries, Sensors and many more....
Include even those concepts, the explanation to which is not very clear even in Apache Airflow's Official Documentation.
Exclusive Features - Data Profiling, Charts, Trigger rules, airflowignore file, Zombies, Undeads, LatestOnly operator.
Learn Best practices / Do's & Dont's to follow in Real-Time Airflow Projects.
Build data pipeline of a Real-Time case study using Airflow.
After completing this course, you can start working on any Airflow project with full confidence.

Add-Ons

Questions and Queries will be answered very quickly.
Airflow codes and datasets used in lectures are attached in the course for your convenience.
I am going to update it frequently, every time adding new components of Airflow.

Who this course is for:

Data engineers who want to deploy their pipelines using Airflow in their company's projects.
Engineers who want to switch their career from conventional schedulers (Oozie, Luigi etc)
Experienced techies who want to add a hot & demanding technology in their resume.

Apache Airflow | A Real-Time & Hands-On Course on Airflow

What you'll learn

Explore related topics

Course content

Introduction4 lectures • 20min

Architecture Of Apache Airflow2 lectures • 11min

Installation3 lectures • 21min

Understanding Directories in Airflow Project2 lectures • 11min

Airflow's UI Tour5 lectures • 37min

What are Operators1 lecture • 4min

Project - Store Sales Report7 lectures • 46min

Airflow CLI Tour3 lectures • 17min

Executors in Airflow4 lectures • 27min

XComs and Variables (Advance)5 lectures • 22min

Requirements

Description

Who this course is for: