Teach on Udemy

Turn what you know into an opportunity and reach millions around the world.

Learn More

Your cart is empty.

Keep shopping

Snowflake - Build & Architect Data pipelines using AWS

Name: Snowflake - Build & Architect Data pipelines using AWS
Rating: 4.5 (635 reviews)

Data engineering and architecting pipelines using snowflake & AWS cloud

Created bySid Raghunath

Last updated 2/2026

English

What you'll learn

Will learn everything needed for Snowpro Advanced Data engineering certification
Snowflake as a data-warehouse & automated pipelines within snowflake ecosystem
Use AWS Cloud with Snowflake as a data-warehouse
Integrating real time streaming data and data orchestration with Airflow and Snowflake

Course content

14 sections • 100 lectures • 8h 47m total length

Course Roadmap2:54
Prerequisites and How to Succeed in this course2:51
Lecture 4 - Feedback and Learn More1:06
Clone Github Repo & PPT for the Course

What is a data-warehouse ?3:23
Two Aspects of a Data Ecosystem2:25
Lab - Setup Snowflake Trial Account2:20
Snowflake Architecture4:01
Discover Snowflake’s cloud-native architecture, a hybrid of shared disk and shared nothing, with three layers: cloud services, compute, and storage, and scalable warehouses with varying pricing.
Snowflake Object Heirarchy4:50
Explore how Snowflake objects are organized in a hierarchy from organization to accounts, schemas, and databases, and how tables, views, stages, procedures, and user-defined functions fit within this structure.
Snowflake - Virtual Warehouses9:48
Snowflake - Different Billing Components9:59
Learn how Snowflake uses credits to bill storage, compute, cloud services, and serverless features, and how to estimate costs across on-demand versus pre-purchased storage and multi-cluster warehouses.
Snowflake - Track your consumption5:23
Snowflake- Resource Monitors4:34

Section Overview2:12
Introduction to partitions and clustering keys7:50
Lab - Micropartitions and Clustering keys17:31
Benefits of Micro-partitions and Clustering5:57
Understanding Clustering Depth and Cluster Overlap9:03
Lab - Selecting your clustering Keys6:08
Lab - Check Query Profile and history4:36
Lab - Query Processing and Caching7:43
Search Optimization Feature10:57
Learn how Snowflake's search optimization accelerates point lookup queries on non-clustered tables by pruning partitions and reducing scans, while understanding cost and maintenance implications.

Section Overview0:44
Data Ingestion - Real World Use Cases4:00
Lab - Create an Integration Object to Connect Snowflake with AWS S37:55
Lab - Ingest CSV from S3 to Snowflake8:27
Learn how to ingest CSV data from S3 into Snowflake by creating a development schema, line item table, file format, stage, and copy into the table.
Lab - Ingest JSON from S3 to Snowflake11:05
Introduction to Continuous Data Ingestion in Snowflake2:01
Lab - Create and implement Snowpipe11:07
Snowpipe - Billing Estimation and Key Considerations for Data Ingestion2:41
Lab - Extracting/Unload Data from Snowflake to S35:33
Learn how to extract and unload data from Snowflake to S3 using a storage integration, with options for partitioned data and file formats, including JSON via object_construct.

Section Overview0:43
Introduction to Streams3:05
Lab - Implement Standard Streams15:02
Lab - Implement Append-Only Streams3:45
Lab - Streams in a Transaction5:46
Streams - Data Retention and Staleness6:03
Learn how Snowflake streams become stale due to data retention limits, time travel, and offsets, and how unconsumed streams affect billing and behavior.
Lab - Change Tracking using "Changes"5:47
Project Overview2:04
Lab - Create Streams - Project Solution Part-17:15
Lab - Create Streams - Part-1 Continuation3:06
Lab - End to End Pipeline in Action5:14

Introduction to User Defined Functions and UDF Types3:18
Lab - Write and implement a Scalar UDF5:07
Lab - Write Tabular UDF in SQL4:02
Lab - Implement Javascript UDFs5:20
What is Pushdown in UDF ?3:25
Understand push down in Snowflake and other big data tools, contrasting load-first filter-later with early filtering. Learn how push down boosts performance and reduces memory use, while recognizing confidentiality risks.
Lab - How can pushdown expose the underlying data ?5:37
Lab - Write Secure UDFs9:36

Section Overview0:48
Introduction to External Functions2:13
Lab - Write Deploy AWS Lambda Function6:36
Create IAM Role2:00
Create an IAM role to grant Snowflake access to database components through the API gateway, naming it currency conversion external role and linking it to the prior S3–Snowflake integration.
Lab - Create API Gateway6:48
Lab - Securing and Deploy API Gateway4:50
Lab - Create External Function in Snowflake6:51

Section Overview1:03
Lab - Connect Python with Snowflake in your local machine3:24
Introduction to AWS Glue2:02
Lab - Deploy and execute python script to AWS Glue5:41
Lab - Parameterize your python script on AWS Glue3:29
Lab - Python Pandas with Snowflake on AWS Glue3:47
What is Pushdown in Spark 3.1 ?3:33
Lab - Deploy a Pyspark script using AWS Glue7:43
Lab - Setup Managed Airflow Cluster on AWS5:54
Set up a managed airflow cluster on AWS for Snowflake data pipelines, including creating an S3 bucket and configuring a CloudFormation VPC with subnets.
Lab - Configure Snowflake Connectivity in Airflow5:51
Lab - Deploy a PySpark Transformation job in AWS Glue4:57
Lab - Setup Airflow DAG4:37
Set up an Airflow DAG to copy data into Snowflake with the Snowflake operator, then trigger a Spark/Glue job, while managing stages, formats, and credentials.

Requirements

Prior programming experience in Sql and python is a must .
Prior basic experience or understanding of cloud services like AWS is important

Description

Course Update as of Feb 2023 : This Course has been updated with Snowpark API which covers UDFs,Stored Procedures for ETL and also covers Machine Learning use-case deployments . This course will help you clear SnowPro Advanced Certifications

Snowflake is the next big thing and it is becoming a full blown data eco-system . With the level of scalability & efficiency in handling massive volumes of data and also with a number of new concepts in it ,this is the right time to wrap your head around Snowflake and have it in your toolkit . This course not only covers the core features of Snowflake but also teaches you how to deploy python/pyspark jobs in AWS Glue and Airflow that communicate with Snowflake , which is one of the most important aspects of building pipelines .

Anyone who has a basic understanding of cloud and belong to one of the below backgrounds can benefit from this course :

- Data Scientists / Analysts

- Data Engineers / Software Developers

- SQL Programmers or DBA's

- Aspiring Data analysts and scientists who are learning SQL and Python

This Course covers :

What is Snowflake
Most Crucial Aspects of Snowflake in a very practical manner
Writing Python/Spark Jobs in AWS Glue Jobs for data transformation
Real Time Streaming using Kafka and Snowflake
Interacting with External Functions & use cases
Security Features in Snowflake

Prerequisites for this course are :

Knowing SQL or at least some prior knowledge in writing queries
Scripting in Python (or any language )
Willingness to explore ,learn and put in the extra effort to succeed
An active AWS Account & know-how of basic cloud fundamentals

Important Note - You need to have an active AWS Account in order to perform tasks in sections related to Python and PySpark . For the rest of the course , a free trial snowflake account should suffice .

Some Tips :

Try to watch the videos at 1.2X speed
Read the reference links and the official documentation of Snowflake as much as possible

Who this course is for:

software engineers,aspiring data engineers or data analyst & data scientists
Also good for programmers and database administrators with experience in writing SQL queries

Snowflake - Build & Architect Data pipelines using AWS

What you'll learn

Explore related topics

Course content

Introduction4 lectures • 7min

Introduction to Snowflake and AWS9 lectures • 47min

Snowflake - Tables6 lectures • 30min

Snowflake - Partitioning , Clustering and Performance Optimization9 lectures • 1hr 12min

Snowflake - Data Loading/Ingestion and Extraction9 lectures • 54min

Snowflake - Tasks and Query Scheduling4 lectures • 15min

Snowflake - Streams and Change Data Capture11 lectures • 58min

Snowflake - User Defined Functions7 lectures • 36min

Snowflake - External Functions7 lectures • 30min

Snowflake with Python,Spark and Airflow on AWS12 lectures • 52min

Requirements

Description

Who this course is for: