Snowflake Master Class: Cloud-Native Data Engineering

Write your first CREATE WAREHOUSE statement with the four settings that matter: size, auto-suspend, auto-resume, and scaling policy. Learn why starting at XS and scaling up beats guessing big from the start.

Understand how multi-cluster warehouses automatically spin up additional clusters when query queues form, eliminating the Monday morning concurrency problem without any code changes.

Learn the production pattern of using separate warehouses for separate workloads — ETL, BI queries, ad-hoc exploration — so one runaway query never kills everyone else's dashboards.

Get a clear mental model of Snowflake's credit system: 1 XS warehouse = 1 credit/hour, with each size doubling. You'll be able to estimate compute cost before you build, not after the bill arrives.

Learn the three types of Snowflake stages (internal, external S3/Azure/GCS, and table stages) and how they act as the loading dock between your source files and your tables.

Write COPY INTO statements that load CSV, JSON, and Parquet files from stages with full control over file format, error handling, and partial load behavior. The command that handles 90% of real-world ingestion.

Automate file ingestion so every new file arriving in S3 or Azure Blob triggers a load within minutes — no cron jobs, no schedulers, no infrastructure to manage.

Store and query JSON, Avro, ORC, and Parquet natively using Snowflake's VARIANT column. Learn dot-notation and colon syntax to flatten nested structures without pre-defining a schema.

Replace nested subqueries with Common Table Expressions that read like a story. Learn how CTEs improve maintainability, enable reuse within a query, and how Snowflake optimizes them at execution time.

Master ROW_NUMBER, RANK, LAG, LEAD, and running totals using OVER() and PARTITION BY. Window functions let you compute across rows without collapsing them — the single skill that separates junior from senior analysts.

Use Snowflake's QUALIFY clause to filter window function results in a single query — eliminating the subquery wrapper that every other database requires. Deduplicate tables and find top-N records in half the SQL.

Read the Query Profile to identify bottlenecks — table scans, expensive joins, spills to disk. Learn how micro-partition pruning works, and when clustering keys improve performance on very large tables.

Create Streams that track every INSERT, UPDATE, and DELETE on any Snowflake table. Learn how METADATA$ACTION and METADATA$ISUPDATE let you process only changed rows — the foundation of efficient incremental pipelines.

Schedule any SQL statement — including MERGE operations — using Tasks with cron or interval schedules. Build multi-step pipelines using task DAGs without a single external dependency.

Define transformation logic once with a SELECT statement and let Snowflake handle the incremental refresh schedule automatically. Dynamic Tables are the declarative alternative to Streams + Tasks.

Compare all three CDC patterns side by side: Streams + Tasks (full control), Dynamic Tables (declarative), and Snowpipe (event-driven ingestion). Walk away with a framework to choose the right pattern for any pipeline.

Design a role hierarchy using USERADMIN, SYSADMIN, and custom roles. Learn how GRANT and REVOKE work, and why the functional role → data role → object privilege pattern prevents the "give everyone ACCOUNTADMIN" anti-pattern.

Mask PII columns dynamically based on the querying user's role — the same table returns real data to authorized users and masked data to everyone else. Add Row Access Policies to filter rows by user or role at query time.

Restrict Snowflake access to approved IP ranges using Network Policies. Learn MFA enforcement, SSO/SAML configuration, and the session policies that reduce exposure without disrupting workflows.

Query Snowflake's ACCESS_HISTORY view to answer any auditor's question: who accessed which columns, when, and from which role. Build the compliance report before the audit — not during it.

Recover from accidental deletes and drops using AT and BEFORE clauses to query or restore historical table states. Learn retention window configuration and the UNDROP command that brings back entire schemas.

Clone tables, schemas, and entire databases instantly using CLONE — no data is physically copied until modifications are made. The safest, fastest way to create dev and test environments from production.

Share live Snowflake data with partner organizations using Shares and the Data Marketplace — no extract, no transfer, no copy. Consumers query your data directly with their own compute, always seeing the latest version.

A decision matrix for all three operational features: when to use Time Travel vs. cloning vs. sharing, with the exact SQL syntax for each scenario. Use this as your daily reference.

Write Python DataFrames using the Snowpark API that execute as SQL inside Snowflake's compute engine — no data movement, no memory limits. The modern replacement for extracting data to pandas.

Call LLMs, run sentiment analysis, generate text summaries, and classify data using Snowflake Cortex functions — all in pure SQL, with no Python or model infrastructure required.

Use Snowflake Notebooks for collaborative Python + SQL development and Streamlit in Snowflake to build data apps — all inside the platform, with no deployment infrastructure.

The five patterns responsible for 80% of surprise Snowflake bills: always-on warehouses, missing auto-suspend, full table scans, over-sized warehouses, and runaway Snowpipe loops — and exactly how to fix each one.

A back-of-napkin framework to estimate your Snowflake costs before a project starts: storage + compute + Snowpipe + data transfer. Stop discovering costs in the finance review and start predicting them in design.

Snowflake Cortex ships pre-built LLM functions you call with plain SQL. COMPLETE generates text, SUMMARIZE condenses it, TRANSLATE converts languages. This lesson shows the syntax, the models available, and the cost model that determines whether Cortex saves you money or wastes it.

Cortex Search creates a fully managed vector search service over your Snowflake tables. No embedding pipelines, no vector databases, no infrastructure. This lesson shows you how to create a search service, query it, and use it as the retrieval layer for RAG.

Beyond generation and search, Cortex ships purpose-built functions for sentiment analysis, text classification, and structured extraction from documents. This lesson covers SENTIMENT, CLASSIFY_TEXT, and EXTRACT_ANSWER — the functions that turn unstructured text into queryable columns.

Retrieval-Augmented Generation combines search with generation. Instead of asking an LLM to hallucinate, you retrieve relevant documents and feed them as context. This lesson builds a complete RAG pipeline on Snowflake: chunk your documents, create a Cortex Search service, retrieve context, and generate grounded answers with COMPLETE.