Snowflake 101 - Build & Design Data pipelines on AWS Cloud

Explore Snowflake's three table types—permanent, temporary, and transient—and learn how persistent storage, session-based data, and time- or row-limited tables support staging and intermediate data.

Learn to create databases, schemas, and various table types in Snowflake, including permanent, temporary, and transient tables, and convert permanent tables to transient to save time travel and failsafe costs.

Learn about simple view, materialized view, and secured view in Snowflake, and how each type shapes data access, performance, and security.

Create standard and materialized views in Snowflake by provisioning a warehouse and selecting database and schema, including an urgent orders view and a secured, clustered materialized view on order date.

Learn how to ingest CSV data from S3 into Snowflake by creating a schema, table, file format, and stage, then copy data and verify 1473 rows.

Ingest json data from S3 into Snowflake by creating a json file format and stage, loading into a raw json table, then transforming into a structured line item table.

learn to create a snowflake role, grant access to a warehouse, database, and schema, and grant select on views, then verify with get DDL and secured views.

Explore Snowflake streams for change data capture (CDC), capturing inserts updates and deletes in raw tables, enabling incremental loads from raw to production, and powering streaming analytics, indexing, and auditing.

Learn to build standard streams for a staging table, ingest data into production, monitor stream activity, update records, and merge changes into the production table.

Demonstrate append-only streams in Snowflake, creating an append-only stream on a raw table, ingesting five rows, and consuming into production while noting offsets and non-capture of updates or deletes.

Learn to connect Python with Snowflake on your local machine using the Python Snowflake connector, install via pip, configure account parameters, run a query, and fetch results.

Deploy and execute a Python script on AWS Glue by uploading a wheel file, setting the Python library path, and running the job in AWS Glue Studio to view logs.

Parameterize a Python script in AWS Glue using getResolvedOptions, passing supplier key and ship date, embedding them in a SQL query, and running the job with parameters.

Learn to run a pandas script locally and on AWS using AWS Glue to query Snowflake data, install pandas, import pandas as PD, and use the data underscore sequel function.

Learn to set up Kafka on your local machine, start Zookeeper and Kafka servers, create a topic, and produce and consume data using console producer and console consumer on localhost:9092.

Configure the Kafka to Snowflake connector by creating and editing sf_connect properties and connect standalone properties, defining topics, topic-to-table mapping, buffering, and Snowflake connection details.

Learn to generate 2048 bit RSA private and public keys for Snowflake, assign the public key to a Snowflake user for Kafka, and verify the key setup.

Discover real-time data streaming with kafka to snowflake, creating a topic, inserting data, and streaming into a snowflake table via kafka connect for end-to-end pipelines.

Learn how snowflake zero copy cloning creates instant, point-in-time copies of databases, schemas, or tables with the clone keyword, without copying data.

Compare clone versus snapshot using a visual, showing how a clone acts as a snapshot and shares micro partitions with the source at time t1, then becomes independent after changes.

Explore zero copy cloning in Snowflake, learn which eight objects can be cloned and what gets cloned automatically when you clone a database or schema.

Learn how to clone a snowflake table with the clone keyword, validate a zero-copy clone, and compare source and clone using time travel and information schema.

Learn to clone transient and temporary tables in Snowflake, including creation, population, and validation of clones with row counts, noting that transient and temporary cannot be cloned to permanent.

Demonstrate object dependency with clone by creating a sequence object and a referencing table, then cloning the table to show zero copy independence within the same schema.

Learn to clone file formats and sequences in Snowflake, create a CSV format, clone it, and observe how sequence increments and next values differ between source and clone.

Demonstrates cloning a Snowflake stream from an existing stream object on an order table, capturing inserts, updates, and deletes as a delta table, with data freshness notes.