Teach on Udemy

Turn what you know into an opportunity and reach millions around the world.

Learn More

Your cart is empty.

Keep shopping

Data Lake in AWS - Easiest Way to Learn [2025]

Name: Data Lake in AWS - Easiest Way to Learn [2025]
Rating: 4.6 (2657 reviews)

Data Lake Mastery: Hands-On Glue, Athena, S3, ETL, Spark, Parquet, QuickSight, Kinesis, Lambda, LLM

Created byChandra Lingam, Compute with Cloud Support

Last updated 10/2025

English

EnglishSpanish [Auto],

What you'll learn

Learn about Data Lake vs. Data Warehouse
Key components of a Data Lake Architecture
Query files directly using SQL
Hands-on integration using Kinesis Firehose, Lambda, Comprehend AI, Glue, Athena and S3

Course content

11 sections • 77 lectures • 5h 59m total length

Introduction to Data Lake12:03
Downloadable Resources0:03
Increase the speed of learning0:41
Storage, Data Ingestion and Migration to a Data Lake3:11
Explore AWS storage options like S3 and Glacier with lifecycle management and real-time ingestion via firehose to S3, Redshift, Elasticsearch, and Splunk, including on-premises migration via Storage Gateway and Snowball.
Data Catalog and Data Swamp2:58
Popular Data Formats6:00
Compare popular data formats for a data lake; understand row vs column storage, Parquet efficiency, and when to convert data to optimize storage and query performance.
Data Transformation2:13
Collect data in its native format and transform in the data lake for analytics. Leverage EMR, Spark, Hive, Glue ETL, and Kinesis Firehose for Parquet or ORC, enabling downstream use.
Udemy Interview - A Deep Dive Into AWS Certifications0:09
Data Lake Introduction

Monitoring and Optimization6:16
S3 Labs0:09
Lab - S3 Storage Classes7:01
Explore Amazon S3 storage classes by creating a bucket, uploading files, and using standard, infrequent access, intelligent tiering, deep archive, and glacier retrieval options to compare costs and retrieval times.
Data Encryption3:12
Security and Protection4:35
Lab - S3 Versioning4:05
Enable S3 versioning to track and manage multiple object versions, read a specific version, delete and undelete objects, and understand storage costs for all versions.
Lab - Age Based Retention (Lifecycle Management)3:50
Lab - Storage Tiering (Lifecycle Management)3:11
Lab - Replication (for disaster recovery)5:45
Set up cross-region S3 replication for disaster recovery by enabling versioning on source and destination, creating buckets in different regions, and configuring a replication rule with an IAM role.
Lab - Encryption with SSE-S3 and KMS6:19
Security and Optimization
Case Study: The 15-Hour AWS Outage and Its Impact on Data Pipelines14:42
From midnight on October 19 to October 20, 2025, AWS’s us-east-1 region faced one of its longest disruptions — lasting over 15 hours and impacting key services like DynamoDB, EC2, Glue and IAM. The incident also triggered cascading failures across other regions that rely on us-east-1–hosted global services.
In this video, we unpack what went wrong, how AWS managed the crisis, and the disaster recovery (DR) strategies every architect should apply to protect workloads from region-level and cross-region dependencies.

Downloadable Resources0:09
Introduction Glue Managed ETL Service and Comparison11:37
Lab - Convert to Parquet Format using Visual Editor8:07
University Rankings Data - Analysis Overview2:28
Explore analyzing a university rankings dataset from Kaggle in a data lake using Athena, performing ELT and ETL with Spark, and visualizing insights with Amazon QuickSight.
Lab - LazySerDe and Issues8:50
Set up a Glue crawler for ranking data and query it with Athena. Use QuickSight for dashboards and address data quality, switching from lazy serde to open csv serde.
Lab - OpenCSVSerDe and Issues5:26
Lab - Cleanup at Query Time (Extract, Load, and then Transform)11:19
Lab - Hide Query Complexity with Views7:42
Glue Jupyter Notebook1:18
Lab - Apache Spark ETL18:39
QuickSight Visualization Labs0:17
Lab - How to Signup for QuickSight 30-day Trial1:59
Instructions to resolve QuickSight - Athena Integration Error0:20
Lab - QuickSight Visualization12:33
Leverage Amazon QuickSight to visualize university rankings by year and region, using Spice data import, pivot tables, and calculated fields to reveal top ranks, distinct counts, and dashboards.
Lab - How to Cancel QuickSight Subscription0:46

Downloadable Resources0:04
How to create PARQUET files?0:18
Handling Schema Changes with CSV Format5:19
Learn how to manage schema changes in a data lake using the glue catalog with csv format, focusing on Athena’s index-based access and risks from shuffled or missing columns.
Handling Schema Changes with PARQUET Format2:29
Learn how Athena reads Parquet by name, supports index-based access, and handles shuffled, missing, or new columns without query changes, noting that renaming columns or data type changes require testing.
Lab - Schema changes with CSV9:27
Learn how csv schema changes impact athena queries and how to preserve column order, add new columns at the end, and keep the glue catalog in sync with crawlers.
Lab - Schema changes with PARQUET5:27
Data Organization Best Practices - Folder Structure, Partitions, Classification10:04
Lab - Partitioned Table Creation and Maintenance7:57
Partition data in S3 and create partitioned tables to query specific partitions, using iris setosa, versicolor, and virginica files to demonstrate partitioning, catalog updates, and performance gains.
Lab - Manual Catalog Maintenance3:15
Lab - Hive Compatible Partitioning4:02
Organize iris data using hive compatible partitions and a self documenting folder structure by type. Create a hive partition crawler, run MSCK and add partitions, and update the Athena catalog.
Partition Projection5:18

Prevent Data Exfiltration with AWS Data Perimeters | Understanding SCPs & RCPs7:29
Learn how AWS's latest security feature strengthens Data Perimeter Security to protect your data from unauthorized access and exfiltration. In this video, I’ll walk you through key strategies to safeguard sensitive data across multiple AWS accounts. Discover how to prevent data leaks caused by misconfigurations, insider threats, and malicious actors.
AWS: Understand the Confused Deputy Problem and 3 Solution Patterns
https://youtu.be/viVoZBc-33s?si=n_415NmIiC2PkGYf

Overview0:25
Lab - Comprehend AI Service for Text Analysis5:36
Explore Amazon Comprehend in a live lab to perform sentiment analysis, custom classification, and medical text extraction, while analyzing syntax, key phrases, and language detection.
Lab - SageMaker Notebook Instance2:49
Downloadable Resources0:03
Lab - Source Code Setup2:25
Lab - Python Introduction6:16
Lambda UI Changes0:14
Lab Part 1 - Serverless Customer Review Application9:45
Lab Part 2 - Serverless Customer Review Application7:51

AWS Cloud Computing and Global Infrastructure0:08
Downloadable Resources0:02
Total Cost of Ownership between On-premises and Cloud0:15
Benefits of Cloud Computing8:39
Explore the six key advantages of cloud computing with AWS, including consumption-based pricing, scalability, speed, global reach, and cost-management tools.
AWS Global Infrastructure10:31
Explore AWS global infrastructure, including public, gov, and China clouds, and see how regions and availability zones, linked by the AWS backbone, ensure data locality and low latency.

Requirements

Basic knowledge of AWS is useful but not mandatory

Description

Hello, my name is Chandra Lingam, and I will be your instructor for the Data Lake in AWS course.

In this course, we will begin by gaining an understanding of the fundamental concepts of a data lake and when it is the appropriate solution as opposed to a data warehouse

We will then delve into the various components that make up a data lake solution, including the ability to query files directly using SQL for rapid ad hoc analysis of datasets

During the course, we will cover the topic of handling changes to the structure of the files in the data lake. We will delve into the various scenarios, such as new fields, new partitions, changes in data types, and missing data, and discuss the techniques on how to handle them effectively. We will also delve into Glue Catalog Management and the evolution of schemas, with a focus on minimizing disruption to downstream systems

We will also look into different data formats, such as CSV, Parquet, Avro, and ORC, and examine their respective strengths and weaknesses. Following that, we will delve into Glue ETL, a robust Apache Spark-based solution for data transformation.

This course is filled with hands-on exercises and projects.

You will analyze a University Rankings dataset, which is easy to understand, useful, and has a mix of data types with many data quality issues.

You will learn to utilize Athena for querying data, tackle data quality problems through SQL, and cleanse the data using Glue - Apache Spark ETL.

Additionally, the course covers techniques for simplifying queries using views and visualizing data using Amazon QuickSight.

To showcase the scalability of Athena, we will query the large Amazon Customer Reviews dataset containing over 130 million reviews. Finally, we will construct a serverless application using Kinesis Firehose, Lambda, Comprehend AI, Glue, Athena, and S3, which can process an unlimited number of customer reviews, perform sentiment analysis, and store the results in the data lake for querying.

I am excited to meet you soon!

Thank you!

Chandra Lingam

Compute With Cloud Inc

Who this course is for:

Decision Makers who want to understand data lake
Professionals who work with structured and unstructured data

Data Lake in AWS - Easiest Way to Learn [2025]

What you'll learn

Explore related topics

Course content

Introduction8 lectures • 27min

Querying and Analytics3 lectures • 10min

Lab - AWS Account and User Setup5 lectures • 21min

Monitoring, Optimization and Security11 lectures • 59min

Lab - Data Catalog and In-place Querying7 lectures • 34min

Glue ETL with Apache Spark15 lectures • 1hr 32min

Schema Evolution11 lectures • 54min

Securing Data with AWS Data Perimeter1 lecture • 7min

Sentiment Analysis Pipeline9 lectures • 35min

Optional: Why Cloud Computing?5 lectures • 20min

Requirements

Description

Who this course is for: