Udemy Business

Teach on Udemy

Turn what you know into an opportunity and reach millions around the world.

Learn More

Your cart is empty.

Keep shopping

Azure Data Engineer Masterclass - The Complete Guide

Name: Azure Data Engineer Masterclass - The Complete Guide
Rating: 4.5 (332 reviews)

Master Azure data Engineer Course with Real Time Projects

Created bymanish tiwari

Last updated 2/2026

English

What you'll learn

Master Azure Data Engineering tools end-to-end with real-time projects
Design and implement robust ETL/ELT pipelines using Azure Data Factory
Write advanced PySpark code for scalable data cleaning and transformations
Work with Azure Databricks notebooks and Delta Lake
Build complete Medallion Architecture pipelines (Bronze, Silver, Gold layers)
Explore Microsoft Fabric: Dataflows Gen2, Pipelines, Lakehouse & Notebooks
Build interactive dashboards using Power BI connected to Fabric/Synapse
Use Azure Synapse Analytics for data warehousing and complex SQL queries
eploy and manage your pipelines using Git Integration & DevOps

Course content

12 sections • 366 lectures • 48h 38m total length

INTRODUCTION0:51
Course curriculum6:54
FILE DOWNLOAD0:01
DEMO - ADF PROJECTS8:14
LAST MINUTE REVISION - NOTES (PDF COURSE)0:01

introduction sql4:50
What is database8:06
Database practical6:05
Insert Statement in SQL7:02
SQL CONSTRAINTS5:58
NOT NULL & UNIQUE CONSTRAINTS5:12
CHECK & DEFAULT CONSTRAINTS4:10
PRIMARY KEY4:51
FOREIGN KEY IN SQL4:41
FILTER & SORT IN SQL5:34
DELETE VS DROP VS TRUNCATE5:53
UPDATE IN SQL4:24
CONDITIONAL STATEMENT IN SQL7:03
AGGREGATE FUNCTION IN SQL3:22
GROUP BY IN SQL5:18
LIKE OPERATOR IN SQL4:50
HAVING IN SQL5:56
TOP OR LIMIT IN SQL5:43
DISTINCT IN SQL1:57
COALESCE IN SQL6:27
JOINING IN SQL13:32
UNION VS UNION ALL6:14
ALTER IN SQL5:40
WINDOWS FUNCTION IN SQL9:46
RANK / DENSE RANK / ROW NUMBER IN SQL10:02
LEAD & LAG IN SQL8:18
CTE IN SQL8:01
VIEWS IN SQL5:55
STORE PROCEDURE IN SQL10:24
SUB QUERY IN SQL5:47
TRIGGERS IN SQL8:12
SQL INTERVIEW QUESTION PART -110:29
SQL INTERVIEW QUESTION PART -211:58
SQL INTERVIEW QUESTION PART -38:45
SQL INTERVIEW QUESTION PART -48:06
SQL INTERVIEW QUESTION PART -58:02

Introdction to adf1:35
create data factory - lab4:16
ADF Components6:04
create storage account8:01
ADF Copy activity14:01
ADF Delete activity5:52
create SQL database6:19
Blob to SQL data copy14:43
TRIGGERS IN ADF4:08
SCHEDULED TRIGGERS IN ADF4:29
EVENT BASED TRIGGERS IN ADF11:14
TUMBLING WINDOW TRIGGERS IN ADF17:17
DATA FLOW IN ADF11:43
JOIN TRANSFORMATION IN ADF10:26
UNION TRANSFORMATION IN ADF8:20
SELECT TRANSFORMATION IN ADF7:31
DERIVED COL TRANSFORMATION IN ADF12:37
AGGREGATE TRANSFORMATION IN ADF8:02
DATASET PARAMETERIZATION IN ADF7:26
LINKED SERVICE PARAMETERIZATION6:13
PIPELINE PARAMETERIZATION11:16
PIVOT TRANSFORMATION9:18
FILE NAMING IN ADF5:49
GET METADATA ACTIVITY IN ADF8:05
FOR EACH ACTIVIY IN ADF9:27
IF CONDITION ACTIVITY IN ADF9:40
SET VARIABLE ACTIVITY6:28
APPEND VARIABLE ACTIVITY2:44
STORE PROCEDRE ACTIVITY8:04
LOOKP ACTIVITY13:49
WAIT ACTIVITY2:34
SWITCH ACTIVITY9:34
VALIDATION ACTIVITY9:36
EXECTE PIPELINE ACTIVITY2:33
FILTER ACTIVITY6:34
UNTILL ACTIVITY10:37
WEB ACTIVITY6:20
FAIL ACTIVITY8:01
SCRIPT ACTIVITY6:33
EXIST TRANSFORMATION8:38
CONDITIONAL SPLIT TRANSFORMATION9:31
PIVOT TRANSFORMATION9:18
NPIVOT TRANSFORMATION8:10
SURROGATE KEY TRANSFORMATION4:54
WINDOWS TRANSFORMATION9:48
ALTER ROW TRANSFORMATION5:34
RANK TRANSFORMATION3:02
ERROR 1- SQL DATABASE V ERROR2:53
EMAIL ALERT SETP IN ADF7:07
IMPORT SCHEMA FAILED ERROR2:23
DATAFLOW IMPORT SCHEMA ERROR6:17
FIND TOTAL FILE COUNT4:55

WHAT IS DATABRICKS8:48
DATABRICKS ADVANTAGE5:06
CREATE DATABRICKS FREE EDITION2:35
EXPLORE DATABRICKS OVERVIEW8:18
WHAT IS DATABRICKS LAKEHOUSE10:41
UNITY CATALOG IN DATABRICKS12:54
CREATE UNITY CATALOG IN DATABRICKS3:15
TABLES IN DATABRICKS - MANAGED VS EXTERNAL TABLES9:47
VOLUMES IN DATABRICKS8:04
CRETAE VOLUMES AND UPLOAD FILES IN VOLUMES8:01
PYSPARK PRACTICAL SETUP - DATABRICKS5:57
CREATE DATAFRAME - DATABRICKS9:40
CREATE DATAFRAME - JSON & PARQUET - PYSPARK5:58
SELECT TRANSFORMATION - PYSPARK4:52
withColumn & withColumnRenamed8:20
FILTER - PYSPARK9:13
DISTINCT VS DROP DUPLICATES - PYSPARK4:58
data = [
(1, "Alice", 23),
(2, "Bob", 34),
(3, "Charlie", 29),
(1, "Alice", 23), # duplicate row
(2, "Bob", 34), # duplicate row
(6, "Alice", 30) # same name, different age
]

columns = ["id", "name", "age"]

df = spark.createDataFrame(data, columns)
display(df)
SORT & ORDERBY3:57
GROUP BY - PYSAPRK4:35
data = [
(1, "Alice", "NY", 2000),
(2, "Bob", "CA", 1500),
(3, "Charlie", "NY", 3000),
(4, "David", "CA", 2500),
(5, "Eve", "TX", 1800),
(6, "Frank", "TX", 2200),
]

columns = ["id", "name", "state", "salary"]

df = spark.createDataFrame(data, columns)
display(df)
JOIN - PYSPARK12:19
UNION - PYSPARK3:26
HANDLE NULL VALUES - PYSAPRK6:41
COLLECT - PYSPARK0:51
STRUCTTYPE & STRUCTFIELD - PYSPARK6:12
PIVOT & UNPIVOT9:00
UDF - PYSPARK8:56
TEMP VIEW - PYSPARK7:07
WINDOWS FUNCTION - PYSPARK15:38
partitionBy ,& Repartition - PYSPARK11:09
DATE FORMAT - PYSPARK5:03
DIFFERENT DATE FUNCTIONS - PYSPARK11:28
EXPLODE - PYSPARK5:30
DATABRICKS FILES8:14
DATABRICKS FLAKEHOUSE FEDERATION5:04
DATABRICKS FLAKEHOUSE FEDERATION - PRACTICAL4:32
WHAT IS DATABRICKS SQL8:47
DATABRICKS SQL - PARAMETERS8:19
DATABRICKS SQL OVERVIEW9:56
DATABRICKS SQL SNIPPETS8:39
DATABRICKS SQL REVISION10:02
SCHEDULING - QUERY4:33
MONITOR - QUERY4:48
DATABRICKS CACHE2:41
ALERTS - DATABRICKS8:00
DATABRICKS - DASHBOARD10:45
GENIE - DATABRICKS8:05
LAKEHOUSE JOBS - DATABRICKS2:50
LAKEHOUSE JOBS - PRACTICAL18:42
IF - ELSE CONDITION - JOBS8:02
FOR EACH CONDITION - JOBS3:58
SCHEDULED & TRIGGER - DATABRICKS3:10
COMPUTE JOBS - DATABRICKS0:59
JOB NOTIFICATION - DATABRICKS1:31
MONITORING - DATABRICKS3:24
DATABRICKS MEDALLION PROJECT - RETAIL SALES INTRO12:14
DATABRICKS MEDALLION PROJECT SETUP & BRONZE LAYER10:59
DATABRICKS MEDALLION PROJECT SETUP & SILVER LAYER17:31
DATABRICKS MEDALLION PROJECT SETUP & GOLD LAYER & DASHBOARD17:14

INTRO TO MICROSOFT FABRIC11:43
HOW TO CREATE FABRIC ACCOUNT14:54
Fabric Hierarchy and terminology6:26
FABRIC OVERVIEW3:45
HOW TO CREATE WORKSPAE IN FABRIC5:07
ONELAKE IN FABRIC8:40
what is lakehosue IN FABRIC10:59
how to create lakehouse in fabric8:10
One lake file explorer in fabric4:24
create tables in lakehose fabric4:29
Shortcuts in fabric5:54
Shortcuts practical in fabric6:19
EXTERNAL Shortcuts practical in fabric10:07
Cache in fabric4:56
sql endpoints in fabric4:14
data factory in fabric8:56
create data factory in fabric6:22
Copy Data Activity - CSV to Parquet8:52
Copy Data Activity - Azure sql to Lakehouse in json8:26
Delete Activity in fabric5:03
getmetadata Activity in fabric12:31
Filter Activity in fabric10:01
Lookup Activity in fabric6:37
Parameterization in fabric8:34
For Each Activity in fabric10:19
If condition Activity in fabric12:48
set variable Activity7:34
Notification in fabric1:30
INVOKE PIPELINE Activity2:34
Schedule pipeline in fabric2:47
Monitor pipeline in fabric3:05
Wait activity in fabric2:54
DATAFLOW GEN2 in fabric8:01
CONNECT SOURCES Using DATAFLOW GEN2 in fabric7:15
DATAFLOW GEN2 -String transformation in fabric3:35
DATAFLOW GEN2 -Mathematical transformation in fabric2:58
DATAFLOW GEN2 -Diagram View in fabric2:46
DATAFLOW GEN2 -Join in fabric3:26
DATAFLOW GEN2 - Load destination table in fabric4:51
Spark Introduction in fabric9:10
Spark pools in fabric3:12
Create your first spark notebook in fabric8:04
Create data frame using pyspark in fabric10:22
Pyspark select , drop and limit command3:13
Typecast in pyspark - with Column & with Column Renamed8:36
Filter in pyspark in fabric4:28
Sort in pyspark3:09
Group By in pyspark3:39
Handle Nulls using pyspark in fabric9:02
Spark SQL IN fabric6:27
Visualization in pyspark notebook fabric8:33
Create Manage and external tables13:03
Notebook utils in fabric notebook9:51
Environment in fabric8:19
Schedule and moniter notebook in fabric4:17
What is Datawarehouse & Usecase7:09
Datawarehouse in fabric8:51
Load the data in warehouse4:11
Copy into command in Fabric datawarehouse4:26
CTAS - Create Table as Select load data in dwh3:19
Views in DWH fabric5:21
Store procedure in TSQL5:34
New visual query in dwh fabric3:02
Download sql database project feature2:01
Write T-SQL in Notebook fabric2:07

WHAT IS POWER BI8:31
POWER BI DESKTOP INSTALLATION2:08
POWER BI DESKTOP OVERVIEW4:42
LOAD DATA IN POWER BI5:52
SUPERSTORE PROJECT - SET HEADER LOGO9:20
SUPERSTORE PROJECT - CARD VISUALS11:14
SUPERSTORE PROJECT- BAR CHART9:48
SUPERSTORE PROJECT - LINE CHART6:22
SUPERSTORE PROJECT - LINE & STACK4:48
SUPERSTORE PROJECT - PIE CHART3:46
SUPERSTORE PROJECT - SLICER5:22
SUPERSTORE PROJECT - STATIC CHART5:22
SUPERSTORE PROJECT - KPI4:20
SUPERSTORE PROJECT - GAUGE CHART2:10
SUPERSTORE PROJECT - SLICER BUTTON2:31
MAP IN POWER BI9:00
Q & A CHART IN POWER BI2:54
TOOLTIP IN POWER BI5:16
FILTERS IN POWER BI8:33
DRILL IN POWER BI2:59
DATA MODEL IN POWER BI7:32
INTRODUCTION TO DAX8:23
DAX FORMAT6:32
DAX DATEDIFF8:05
DAX SWITCH CONDITION5:24
DAX VARIABLE5:37
DAX CALCULATE & FILTER5:56

INTRODUCTION - PYTHON4:41
installation anaconda3:09
WHAT IS PYTHON & FEATURE7:17
PYTHON DATA TYPES & VARIABLES3:56
INPUT FUNCTION - PYTHON6:26
OPERATORS - PYTHON5:05
IF -ELSE STATEMENTS - PYTHON14:59
FOR LOOP - PYTHON10:38
WHILE LOOP - PYTHON4:44
LIST IN PYTHON8:56
LIST METHOD8:25
LIST CODING PROBLEM(LOWEST & HIGHEST) -14:22
LIST CODING PROBLEM -2 (SWAP 2 ELEMENTS)3:25
LIST CODING PROBLEM -3 (REVERSE LIST)2:24
LIST CODING PROBLEM -4 (SUM& AVG LIST)2:46
LIST CODING PROBLEM -5 (EVEN & ODD LIST)2:07
LIST CODING PROBLEM -6 (REMOVE DUPLICATE LIST)1:21
LIST COMPREHENSION6:21
TUPLES IN PYTHON10:07
SETS IN PYTHON8:14
dictionary in python10:31
STRING IN PYTHON8:54
string coding7:04
FUNCTIONS IN PYTHON11:35
ARGS & KWARS IN PYTHON5:26
LAMBDA FUNCTION IN PYTHON4:54
MAP , FILTER & REDUCE IN PYTHON10:43
OBJECT ORIENTED PROGRAMMING8:09
CONSTRUCTOR & METHOD IN PYTHON8:35
INHERITENCE IN PYTHON5:52
ENCAPSULATION IN PYTHON6:47
ITERABLE VS ITERATOR4:32
MODULE IN PYTHON8:29
OS MODULE IN PYTHON6:58
DATE AND DATETIME IN PYTHON8:21
TRY - EXCEPTION IN PYTHON6:42

DATA ENGINNERING PROJECT - ADLS + ADF + DATABRICKS +SPARK SQL44:59
AZURE DATA ENGINNEING PROJECT - ADF + ADLS + AZURE SQL + DATABRICKS + POWER BI1:12:59
AZURE DATA ENGINEER PROJECT - ADLS + ADF + SPARK + SQL - SYNAPSE46:06
END TO END MICROSOFT FABRIC PROJECTS40:06
https://github.com/manish040596/FABRIC-PROJECTS/tree/main/SALES%20ANALYTICS%20PROJEC%20T-%20FABRIC%20DATASET
DATA MIGRATION - ONPREM SQL TO AZURE STORAGE16:53
DATA PIPELINE PROJECT - INCREMENTAL LOAD36:21

Requirements

Practice is must

Description

Become a Job-Ready Azure Data Engineer
Master real-world data engineering with this hands-on, beginner-to-advanced course designed for aspiring and working professionals.

What You Will Learn:

Azure Data Factory (ADF):

Design & orchestrate ETL/ELT pipelines
Integrate data from SQL, Blob, REST APIs & more
Build Data Flows and automate with Triggers

Azure Databricks & PySpark:

Process big data using PySpark on Azure
Implement Delta Lake & optimize Spark jobs
Real-world notebook-based projects

Azure Synapse Analytics:

Create and query data using Dedicated & Serverless SQL Pools
Use Synapse Studio for data transformation & analytics
Integrate Synapse with ADF and Databricks

SQL for Data Engineering:

Write efficient SQL queries for transformations
Use joins, window functions, and aggregations
Practice with real datasets and assignments

Python for Data Engineering:

Automate tasks and create clean, modular scripts
Use Python in ADF and Databricks notebooks

Real-World Projects + Interview Prep:

Industry-based mini projects for each topic
Azure Data Engineer interview questions and answers
Resume & career guidance included

Who Should Take This Course?

Aspiring Data Engineers & Cloud Developers
SQL Developers transitioning to cloud data platforms
Anyone preparing for Azure Data Engineer Associate
Professionals looking to gain hands-on Azure data stack skills

Tools Covered:
Azure Data Factory, Databricks, Synapse Analytics, SQL Server, Blob Storage, Delta Lake, Python, Git, Event Hubs, and more.

No prior cloud experience required! All concepts are explained from scratch.

Who this course is for:

DATA Enginner enthusiast
Azure Data Engineer enthusiast
Beginner Looking for Data Engineer role

Azure Data Engineer Masterclass - The Complete Guide

What you'll learn

Explore related topics

Course content

Introduction5 lectures • 16min

COMPLETE SQL36 lectures • 4hr 7min

COMPLETE AZURE DATA FACTORY52 lectures • 6hr 40min

COMPLETE DATABRICKS WITH PYSPARK - NEW58 lectures • 7hr 21min

Azure Synapse Analytics10 lectures • 1hr 18min

MICROSOFT FABRIC65 lectures • 7hr 1min

COMPLETE POWER BI27 lectures • 2hr 42min

COMPLETE PYTHON36 lectures • 4hr 3min

AZURE DATA ENGINEER PROJECTS6 lectures • 4hr 17min

GIT CICD1 lecture • 15min

Requirements

Description

Who this course is for: