Teach on Udemy

Turn what you know into an opportunity and reach millions around the world.

Learn More

Your cart is empty.

Keep shopping

2025 Mastering dbt (Data Build Tool) - From Beginner to Pro

Name: 2025 Mastering dbt (Data Build Tool) - From Beginner to Pro
Rating: 4.7 (1285 reviews)

Hands-on Analytics Engineering Bootcamp With: Theory, Building a dbt Project from Scratch, and Deploying to dbt Cloud

Created byJack Colsey

Last updated 8/2025

English

English [Auto],Spanish [Auto],

What you'll learn

How to build a complete dbt project from scratch
The main benefits of dbt, and a bit of background as to how it came about
All of the dbt fundamentals: sources, models, tests, documentation, snapshots, seeds, macros, hooks, and operations
How to structure a dbt project: staging, intermediate, and mart models - and naming conventions
How to version control changes to your code with GitHub and VSCode
Advanced dbt testing - creating your own custom singular & generic tests, setting severity, and setting warn/error thresholds
Advanced dbt data modelling - model materialisation and governance (access, contracts, and versions)
Advanced dbt commands - how to use different selectors, different profiles, tags, indirect test selection and building a local dbt documents site
Advanced dbt jinja & macros - creating your own macros to use in hooks / functions / operations, using jinja for loops and variables, and the target function
How to deploy your project on dbt Cloud, how to use the dbt Cloud UI, and using environment variables
How to use tests & macros from external packages to supercharge your dbt project
Best practises to use when running a dbt project (based on lots of experience!)
How to create a complete setup for Mac or Windows: installing all of the tools and getting a dbt specific VSCode setup!

Course content

8 sections • 110 lectures • 7h 51m total length

Instructor Introduction1:31
Course outline2:32
Course Introduction3:17
Kick off the 2025 mastering dbt course with a hands-on introduction to building a dbt project from scratch, documenting and testing SQL models, and leveraging macros and jinja.
A Brief History of the Data Stack2:42
Trace the data stack's evolution from raw data and manual metrics to data warehouses and automated pipelines, highlighting shared metrics, run order, data quality, and documentation challenges.
Benefits of dbt - Inferring Dependencies4:31
Benefits of dbt - Documentation & Testing4:04
Understand how dbt uses yaml documentation to test sql data models, describing columns and constraints, and generate a docs site that shows model lineage and upstream and downstream relationships.
Benefits of dbt - Python-Like Functionality6:56
Slight correction here at 4:30 - it should be all_values_equal_to, not test_all_values_equal_to!
How dbt Has Solved a Lot of Problems in the Data Stack2:11
Discover how dbt reduces data stack pain by inferring model run order, simplifying documentation and testing, and enabling python-like functionality with ginger for richer sql workflows.
How dbt Fits in the Data Stack4:11
See how dbt fits the transform step in the ELT data stack, managing dependencies, templated SQL with Jinja, and automated testing and documentation for databases like BigQuery and Snowflake.
dbt Core vs. dbt Cloud2:04
Compare dbt core and dbt cloud, highlighting core’s CLI-based development with databases like Redshift, Snowflakes, and BigQuery, and cloud’s web editor and job scheduling, with the course focus on core.
Section Recap0:25

Section Overview1:04
Set up your dbt workflow with Google BigQuery, Python, Visual Studio Code, and GitHub, establishing required Gmail accounts and tools to build projects.
Note on Continual Course Updates1:57
Update this course to reflect new dbt versions, rerecording only when significant changes occur, and keep older content relevant to newer releases.
Help If You Get Stuck During This Course0:33
Creating a Gmail Account0:50
Create a Gmail account by navigating to accounts.google.com/sign up, entering required details, reviewing terms, and clicking agree.
Setting up a BigQuery Project With Billing2:26
Set up a Google BigQuery project with a free trial, learn on-demand analysis, and understand the first terabyte free and pay-per-use billing.
(Optional) If You Have Issues With BigQuery Billing2:04
Resolve BigQuery billing issues by adding and verifying a billing account in console.cloud.google.com, link it to your project, and upgrade the billing banner.
The BigQuery UI4:07
Master the BigQuery UI by setting up a billing project, using the public look e-commerce dataset, and running sample SQL queries in the editor within the free tier.
The Dataset You'll Be Using3:01
(Mac) Installing Python 3.101:11
(Windows) Installing Python 3.100:57
Downloading VSCode and Setting Up Shortcuts5:23
Creating a GitHub account2:25
Create a GitHub account to store and version control your dbt projects, enabling collaboration and remote backups. In the next video, pull code from the remote repository.
Forking Vs. Cloning2:29
Compare forking and cloning on GitHub, and learn how forking your own copy lets you push changes while cloning only pulls to your computer.
Forking the Repository2:21
Fork the tutorial course repository on GitHub and clone the fork locally using VS Code to start working. Then install the recommended extensions to complete the setup.
(Optional) If You Have Issues Syncing Your Forked Repository4:25
Installing the recommended VSCode Extensions0:45
Install the recommended VSCode extensions for dbt, clicking install on three of four and awaiting the fourth before enabling, including syntax highlighting, a non-version-controlled add-on, and yaml formatting.
What's a Virtual Environment (venv)?2:53
Setting Up Our Virtual Environment and Installing Packages5:58
Create and activate a virtual environment on macOS, Windows, or Git Bash, then install required Python packages from a requirements.txt file to standardize your workspace.
Setting Up dbt for BigQuery3:54
Set up dbt for BigQuery by auto-activating a Python virtual environment and configuring gcloud authentication to run dbt commands against BigQuery.
Trialling Our Model dbt Project6:24
(Optional) Setting Up dbt Autocomplete2:20
Run Through of How Our Final Project Will Look10:18
Section Recap0:13

Section Overview0:13
The dbt init Command4:48
Commands used:
dbt init
dbt debug --config-dir
Version Control with GitHub6:15
Setting up dbt Power User6:52
Set up the dbt power user extension and configure the profiles directory and profiles.yaml for your system. Preview and run compiled sql to inspect downstream and upstream models.
How We'll Structure Our Project6:20
Structure a dbt project with staging, intermediate, and marks folders, using one source per staging table, clean yaml separation, and naming conventions like stg_<source>__<table> and dim or fct tables.
Creating Our First Source (src) yml File5:05
(Windows) Issues with the dbt Power User extension1:03
Creating Our First Staging (stg) SQL Model7:04
Running Our First Staging (stg) SQL Model4:58
Creating Our First Model yml File3:37
Create a model YAML file for your data model using the codegen macro and dash run to generate a template, with model_names and a stage e-commerce orders example.
Adding Tests to Our First Model yml File8:26
Setting Up Our Models to Materialise as Tables Instead of Views4:16
Configure dbt to materialize models as tables by default using yaml, resource paths, and project-wide settings. Compare table and view behavior in BigQuery and per-file overrides.
Getting the Rest of Our Staging (stg) SQL Models Set Up3:51
Using dbt clean to Get Table Materialisation Working1:29
Getting the Rest of the Staging (stg) yml Files Set Up5:46
Taking Stock of Our Staging (stg) Data Models3:32
The Target Folder5:48
Getting Our First Intermediate (int) SQL Model Set Up7:28
Getting Our First Intermediate (int) yml File Set Up7:29
Getting Our Mart SQL Model Set Up8:29
Getting Our Mart yml File Set Up3:58
Create and configure the orders.yaml dbt model file, align the file name with the model, adjust for advanced parts, and run tests to verify integration with BigQuery.
Our Basic dbt Project Is Now Complete!1:29
Section Recap0:21

Section Overview0:16
Setting Default Test Severity4:14
Set default test severity to one under the tests tab in the lessons project, under the models tab, and set expiration to 168 hours (seven days) to avoid stale tables.
Setting Test Severity and Thresholds8:22
Configure dbt tests by setting severity to warn or error, and see how the pipeline reacts to each setting while validating data quality on the stage ecommerce products model.
The External dbt Packages We'll Be Using8:12
dbt_utils and dbt_expectations6:31
Learn to use dbt_utils and dbt_expectations to validate cost and retail price in stage e-commerce products, adding min zero constraints, expression true checks, and named tests for clear feedback.
Custom Singular Tests8:24
Discover how to create custom singular tests in dbt using SQL checks that return rows to flag data mismatches, like order item counts versus orders, with configurable severity.
Custom Generic Tests8:08
Explore building custom generic tests in dbt by creating a generic folder, defining a primary key test in yaml, and using jinja templating for not null and unique validations.
Applying Advanced Tests to Our Whole Project3:26
Source Freshness Tests4:03
Unit Tests8:49
Section Recap0:26

Section Overview0:21
The doc Function5:28
Seed Files6:33
discover how to use seeds, version controlled csv files, to load new data into your warehouse with dbt; create seeds and seeds.yaml, validate column types, and load before downstream runs.
dbt Snapshots11:00
Materialisation Types7:55
Materialisation: Ephemeral Models8:36
Materialisation: Incremental Models12:42
Master incremental models in dbt by building tables with new rows only, using the incremental macro, drop and replace behavior, a unique key, and full refresh or schema changes.
Materialisation: Microbatch Incremental Models14:36
Explore micro-batch incremental models in dbt, replacing whole partitions by day/month/year, enabling selective backfills and reruns of specific data from event time, with begin, batch size, and event time configuration.
(Optional) Partitioning a Table in BigQuery3:43
Model Governance Overview2:15
Explore data model governance in dbt, covering model access, model contracts, and model versions to enable scalable dbt projects and safer, controlled releases.
Model Governance - Access & Groups5:50
Model Governance - Contracts8:00
discover how data contracts advance model governance by enforcing that every column has a defined data type and constraints, checked before model creation via YAML in dbt.
Model Governance - Versions10:22
Section Recap0:31

Section Overview0:23
Commands For a Clean dbt run3:43
Using Different dbt Profiles4:53
learn to run dbt with multiple profiles by using target flags and a profiles.yaml in a profiles directory, directing the same project to dev, testing, or production, including cloud setups.
Selectors4:59
Tags2:14
Indirect Test Selection5:32
dbt test With --warn-error2:28
dbt build1:30
dbt docs generate / serve3:33
The "Empty" Flag1:35
Explore the dbt empty flag that runs SQL without processing data, creating empty tables to test code quickly and cheaply while verifying SQL behavior.
Section Recap0:36

Section Overview0:21
Jinja Comments, Statements, and Expressions8:05
The 3 Types of Macro: Functions, Hooks, Operations4:13
Learn the three macro types in dbt: functions, hooks, and operations, and how to build reusable sql logic with is weekend macros, before and after run hooks, and run operations.
(Optional) dbt Jinja Function Reference5:35
Explore built-in dbt jinja functions and learn how to leverage adapter.get_columns_in_relation to list model columns. Use dbt_utils.get_column_values to fetch distinct column values, such as order status, from existing tables.
Macros: Operations8:39
Macros: Functions (Building a Basic Macro)6:23
Macros: Hooks5:59
Discover how hooks operate before and after models and runs in dbt, using macros to create reusable functions and orchestrate pre and post run actions.
Jinja Statements: for Loops and Setting Variables6:17
Explore jinja statements for for loops and set variables to pivot column data by department in SQL. Learn to build dynamic, department-based total sold columns using the get_columns utility.
(Optional) Jinja: Using the Target Function5:23
Section Recap0:29

Section Overview0:14
This section focuses on everything to do with dbt Cloud. In this section we will:
Create a dbt Cloud account
Configure dbt Cloud to connect to our dbt project
Understand how to use the dbt Cloud IDE (think of it as being a web version of VSCode)
Deploy (schedule) our dbt project as a pipeline on dbt Cloud
Creating a dbt Cloud Account0:35
Setting Up a Service Account3:18
Create and configure a service account in BigQuery, generate a JSON key, and upload it to dbt to enable credentials for running jobs and testing the connection.
Connecting GitHub to dbt Cloud1:20
The dbt Cloud IDE4:44
Explore configuring a project, managing branches, viewing lineage, running and testing models, and using pre-configured commands in the dbt cloud ide.
Deploying Jobs on dbt Cloud8:22
Deploy dbt Cloud pipelines by creating production deployments, configuring environments, and scheduling jobs with source freshness and docs generate and docs serve, while managing credentials and BigQuery permissions.
Section Recap0:17

Requirements

Basic SQL
No Python experience needed
Mac / Windows machine which is capable of installing Python, Git, and VSCode (we'll run through all of this in the course!)

Description

A complete course to help anyone with basic SQL skills learn advanced dbt, a key tool for Analytics Engineering!

Welcome to the 2025 Mastering dbt (data build tool) course! This course runs through everything from the theory behind dbt to building an advanced dbt project (from scratch) and deploying it on dbt Cloud.

I have over 9 years of experience across Analytics / Analytics Engineering / Data Science, including 5 years using dbt on a daily basis. I was also involved in the rollout of dbt in my time at Monzo Bank!

In this course I've taken everything I've learnt over the past 5 years, and what I use on a daily basis, and condensed it to take anyone who knows SQL to an advanced level of dbt as quickly as possible.

MY APPROACH TO THIS COURSE:

We'll cover everything you need to know about dbt: from the basic data modelling right through to all of the advanced features such as creating custom tests and macros. We'll be doing this step by step, and build from the basics upwards.

It's focused on practical outcomes - we won't be spending ages on database theory, or going into lots of detail on the eCommerce dataset we'll be using, instead we'll be aiming to get you up to advanced dbt levels as quickly as possible.

For every video where we're writing code, I've created lesson attachments with the final outputs. This means you can either code as you go along, or watch the videos and look at the handouts afterwards! I've also included some theory with these handouts to help hammer home the points made in the videos.

There's also a public GitHub repository (which you'll be using for this course) that contains a model final project you can reference throughout.

This course isn't static! I'd love to hear your feedback and will be updating this course on an ongoing basis.

COURSE STRUCTURE:

This course focuses on first getting a good understanding of what problems dbt solves, then building a basic dbt project, before layering on more advanced concepts and finally deploying our project with dbt Cloud.

Introduction
Some theory (<1 hour) around dbt, what problems existed in the data stack before it came along, and how it solves them.
Tool setup
Getting set up with Python, GitHub, Google BigQuery, VSCode, and of course dbt! If you're familiar with any of these tools already then you are more than welcome to skip the appropriate lessons.

We'll also be exploring the fictional eCommerce dataset that we'll be using throughout the course.
Building our basic dbt project
This section focuses on creating our project from scratch, including how we will structure our project.

We'll be building out staging (stg), intermediate (int), and mart data models, including documentation & testing with the out-of-the-box dbt tests.
Advanced dbt testing
We'll start to build on our basic dbt project by setting test severity & thresholds, using the dbt-utils and dbt-expectations external packages for their excellent selection of tests, creating our own custom singular & generic tests, unit testing, and testing the freshness of our source data.
Advanced data modelling with dbt
Next, we'll be looking at how we can create reusable documentation, seed files (version controlled .csv files), snapshots (capturing changes to data tables), and materialisation methods.

Most of this section will be focused on the last part - the materialisation methods: ephemeral, view, table, and incremental (including microbatch). By this point we'll have encountered view & table models and we will be building both an incremental and an ephemeral model - and you will gain an understanding of what to use and when.

This section includes all model governance features from dbt version 1.5! This includes model access, groups, contracts, and versions.
Advanced dbt commands
This section will focus less on changing our dbt project, but instead all of the major dbt commands and how (and when) to use them.
Advanced Jinja & macros
The final changes to our project will involve using Jinja - a core feature of dbt and arguably it's most complex but powerful feature - and using it to create our own macros.

This section will run through how you can use Jinja macros for hooks, operations, and as reusable functions in your SQL models. It'll also run through some theory around Jinja, common mistakes, and what I (personally) find to be what it's most useful for!
dbt Cloud
Finally, we'll be exploring how to take our project and deploy it on dbt Cloud - including how to schedule it to run on a regular basis. We'll also be looking at dbt Cloud itself and its main benefits.

Who this course is for:

Data Analysts
Data Scientists
Analytics Engineers
Data Engineers
BI Professionals
Anyone interested in getting into data!

2025 Mastering dbt (Data Build Tool) - From Beginner to Pro

What you'll learn

Explore related topics

Course content

Introduction11 lectures • 34min

Getting Set Up with Your Tools23 lectures • 1hr 8min

Building the Basic dbt Project23 lectures • 1hr 49min

Advanced dbt: Testing11 lectures • 1hr 1min

Advanced dbt: Data Modelling14 lectures • 1hr 38min

Advanced dbt: Commands and Selectors11 lectures • 31min

Advanced dbt: Jinja and Macros10 lectures • 51min

dbt Cloud7 lectures • 19min

Requirements

Description

Who this course is for: