Mastering Google Dataform: From Beginner to Pro- BigQuery

Create a GCP project and enable billing to access data form services, including a free $300 credit for three months, and set up a GitHub repository with version control.

Learn dataform initialization by creating a development workspace, a first branch, and pushing definitions, graphs, and the workflow settings YAML to GitHub.

Set up and explore a Google Dataform project, choose browser-based or local development, integrate with GitHub, push branches, and prepare a first model to transform data in BigQuery.

Install Visual Studio Code, clone the repository from GitHub into your data form project, then initialize the data form CLI in a terminal.

Create a service account, grant editor access to BigQuery resources, generate and rename a JSON key, configure dataform credentials, add the key to gitignore, and test compilation.

Delete unused Dataform models, keep the first to test the real model, create a branch, and use BigQuery to create an EU data set and configure Dataform for modeling.

Learn to create and configure a simple Dataform model, run it locally with the Dataform cli or service, and verify tables in BigQuery datasets such as data form and raw.

Create and configure a drinks data source in dataform for BigQuery, declaring the database, schema, and table, and connect it to dependent views to simplify updates and tracking.

Create a BigQuery view for the drinks table selecting drink id, drink name, and drink type with distinct when needed; name it view_drinks, then replicate the same in data form.

Explore how incremental models handle daily data by updating existing records and adding only new ones, using the id to avoid duplicates and keep data up to date for analysis.

Explore implementing incremental models in Dataform, learn to run a single model via CLI actions, and build incremental tables with a combined drink-id and meal-id key using joins in BigQuery.

Explore the slowly changing dimension model and learn how to implement it in Dataform by tracking historical changes with validity dates and inserting new records instead of updates.

Organize dataform definitions by creating a slowly changing dimensions folder and switching to data format JSON. Build a meals SCD with a source data CD, a view, and BigQuery table.

Learn how to implement assertions in dataform models to enforce data quality, catch nulls, and trigger alerts or pipeline failures when data violates rules, using drink data examples.

Push local changes, create and merge development branches into main via pull requests, resolve conflicts, and set service account permissions to run models in dataform.

Navigate the dataform graph that reveals lineage and dependencies among models, views, and assertions, using references for scalable pipelines; run tasks with full refresh and fix IAM permissions if needed.