Snowflake - Complete End To End Data Project Hands-On Guide

Name: Snowflake - Complete End To End Data Project Hands-On Guide
Rating: 4.5 (628 reviews)

Master Snowflake Complete E2E Data Project + Data Pipeline Hands-on Guide With Live Data From Gound Up

Created byDE & AI Engineering Simplified

Last updated 9/2024

English

What you'll learn

How to architect a Snowflake based data engineering project
How to design end to end flow with Snowflake based data engineering project
How to load and ingests API data into snowflake
How to develop an end to end data engineering project with Snowflake
How to automate API data ingestion in Snowflake
How to use stages, task, dynamic tables to create a DAG in Snowflake.

Course content

6 sections • 31 lectures • 4h 6m total length

Introduction - E2E Data Project5:18
Master an end-to-end data engineering project in Snowflake, from API JSON extraction to cleansing, transformation, and star schema modeling, with Streamlit dashboards.
Technical Pre-requisite2:17
Course Structure3:35
Explore the six-part course structure, translating business use cases into requirements, modeling data flows, building end-to-end pipelines, and delivering streamlit-powered dashboards and real-time analytics with Snowflake.
Data Engineering Use Case4:15
Resources - Architecture Diagram, SQL Scripts, Snowpark Code & Data Files1:25

Create Dabase, Schema & Warehoues8:05
Load Data Files To Stage Location16:17
Flatten & De-Duplication (Cleansing)13:24
Transpose The Data (Rows To Columns)7:14
Design & Implement Wide Table Approach6:42
Design & Implement Fact + Dimensiont Tables9:56
Construct date and location dimension tables and a fact table using a hash-based key within the consumption layer, enabling dimensional modeling of AQI measurements.
Load Additional Data & Observe Data Flow3:40
Aggregated Fact Table (Design & Implementation)8:46
Design and implement aggregated fact tables at city level to summarize station-level air quality index data, compute city-level aqi averages, and enable hourly trends via common table expressions.
Day Level Aggregated Fact Table3:34
Large Data Loading & Validate Data Pipeline15:56

How to add data marketplace in snowflake6:36
Creating Dynamic Table On Data Marketplace Views & Challenges1:54
Creating Standard Table & Task To Consume Marketplace Data3:52
Create a Snowflake standard table and a daily refresh task to ingest marketplace weather data, build an aggregated daily fact table, and enable hourly and daily data consumption via dashboards.
Streamlit Dashboard With Snowflake Marketplace Data3:18

Requirements

Familiarity with Snowflake Cloud Data Warehouse
Basic understanding of SQL queries using Snowflake
Basic understanding of python programning
Basic understanding of Github version control system

Description

Want to learn how to architect, design and implement End To End Data Project in Snowflake?

Do you want to upgrade your Snowflake skill and learn how to build complete end to end data project using Snowflake Cloud Data Warehouse Platform?

In this course, you will learn everything, starting from source file analysis, evaluating different architectural issues to ingetrate data, layered architecture to store different datasets, design stage/raw/curated layers and finally automate your data pipeline and implemente fully automated data flow from source to visualization with real life.

Why should you take this course?

Understand everything about end to end data engineering project - a step by step guide
Different data integration challenges and how to build a solution to ingest data into snowflake.
How to design a layered architecture to store and process the data.
How to automate the data integraiton to ingest live data.
How to automate and create a fully functional DAG.
How to use Snowflake dynamic tables to create fully functional data pipeline.
How to create a visualization using streamline within Snowflake
How to use Snowpark Python library with GitHub action to automate data flow.
How to use free marketplace data and integrate it with Snowflake data pipeline.

Who this course is for:

Snowflake Cloud Data Developer
Data Engineer
Snowflake Data Engineer

Snowflake - Complete End To End Data Project Hands-On Guide

What you'll learn

Explore related topics

Course content

Introduction - Snowflake End To End Data Engineering Project5 lectures • 17min

Snowflake Data Project - Design & Architecture4 lectures • 56min

Building End To End Data Pipeline - Snowflake10 lectures • 1hr 34min

Visualization Using Streamlit Within Snowflake4 lectures • 36min

API Data Source Integration With Snowflake4 lectures • 28min

Snowflake Data Marketplace4 lectures • 16min

Requirements

Description

Who this course is for: