
I am explaining course structure
Module one of the snowflake masterclass covers architectures, basics, virtual warehouse, and performance optimization. Learn environment setup, caching, clustering, dashboards via the user interface, and query acceleration and search optimization.
Create a Snowflake trial account, select the business critical edition on AWS, choose a region to minimize data transfer costs, and log into the Snowflake web console.
Learn to create a Snowflake database named demo db, explore the information_schema and public schema, and build an employee table with sample data for practical demos.
Explore Snowflake architecture, including data storage, virtual warehouses, and cloud services on AWS S3, and how time travel and zero-copy clone enable cost-efficient query processing.
Explains shared nothing architecture by contrasting it with a shared disk bottleneck and allocating storage per compute node. Discusses data shuffling, node failure impact, and storage–compute balance challenges.
Revisit Snowflake architecture by examining data storage and compute separation, coordinated by the cloud services layer that submits queries to the virtual warehouse in a scalable multi-cluster setup.
Explore demos to understand Snowflake architecture better and see how data processing happens under the hood. Learn how architecture layers interact and how caching works in Snowflake.
Explore Snowflake caching mechanisms across cloud services, virtual warehouses, and data storage layers, showing how query results are reused via result cache and local disk caching.
From this architecture demo, learn to use a virtual warehouse for queries, avoid select star on large tables, set a limit, and adjust auto suspend.
Learn how snowflake's cloud services cache is invalidated by updates and when results are pulled from storage, with cached results saved for 24 hours and shared across users.
Explore clustering in Snowflake, learning how clustering works under the hood and how micro partitions are created, with demos and insights into query processing, dos and don'ts, and potential drawbacks.
Submit a query to Snowflake, where cloud services optimize the plan, assign work to virtual warehouses, prune micro partitions by metadata and filters, and enable selective column loading with clustering.
Discover micro partitions in Snowflake and how loaded data forms immutable table files. These partitions use columnar storage with PACs compression and include headers with column offsets and min/max values.
Explore micro partition depth in Snowflake by examining overlapping partitions, how depth drives clustering decisions, and the concept of constant micro partitions.
Create a clustering key on a Snowflake table by segment to enable automatic micro-partition grouping, then extend with additional keys using substring and to date conversions.
Learn to check a table's clustering information using Snowflake to view micro partitions, depth, and distribution, and evaluate clustering keys and potential reclustering impacts.
Demonstrates how clustering by segment reorganizes micro partitions after data load, using clustering information to watch depth in real time and understand backend compute costs.
Discover how to improve Snowflake performance by loading data with an order by on key columns, avoiding clustering operations and saving costs on micro partitions.
Learn how to create and configure Snowflake virtual warehouses, including standard vs snowpark optimized types, auto scaling with min and max clusters, and the maximized mode.
Learn how Snowflake scales virtual warehouses with multi-cluster policies, including standard and economy modes, deciding when to spin up new clusters and when to suspend idle ones.
This course is the result of my over 4 years of experience working with Snowflake. Snowflake is a fantastic tool to handle huge volumes of data. You can load, query and export, and run transactional queries on TB's and PB's of data easily.
You don't have to worry about infrastructure setup, security, performance, tuning, Garbage collection, node failures etc etc...
if you have already worked with Spark with EMR or with Data bricks you will be amazed by snowflake performance and simplicity.
But there is a catch, if you are not fully aware of how things works in snowflake, you will easily loose control and end up writing costly queries and end up using many of snowflake features wrongly.
I mentioned costly queries because snowflake is build for performance.
Costly queries and wrong design decisions in snowflake will sky rocket your snowflake bill. I have made lot of such mistakes and learned from those mistakes.
This course is an effort to walk you through such experiences so that you don't end up doing same mistakes which i did!!!
Few things can only be learned by experience you can't find them in documentations this course will expose such details to you.
Hope you join me in this journey
High level overview of all modules with sections details is given below,
In module#1,
> Snowflake environment setup.
> Snowflake architecture.
> Snowflake caching , micro-partitions, clusters.
> Snowflake clustering deep dive.
> Snowflake virtual warehouse layer.
> Performance tuning and DB characteristics.
In module#2,
> Loading data in snowflake.
> Loading data using internal stage.
> Load data from aws s3.
> Load data from google gcs.
> Load data from Azure.
> Copy command options.
> Working with copy command.
> Assignments.
> Loading unstructured data
In module#3,
> Snowpipe in snowflake.
> Data sharing in snowflake.
> Time travel.
> Fail safe.
> Clone feature.
> Data sampling.
> Tasks in snowflake.
> Streaming in snowflake.
> Continuous data load.
> Materialized views.