Google Cloud Professional Data Engineer: Get Certified 2022

Name: Google Cloud Professional Data Engineer: Get Certified 2022
Rating: 4.3 (4700 reviews)

Build scalable, reliable data pipelines, databases, and machine learning applications.

Created byDan Sullivan, Dan Sullivan

Last updated 5/2025

English

Czech [Auto],English [Auto],

What you'll learn

How to pass the Google Cloud Professional Data Engineer Exam
Build scalable, reliable data pipelines
Choose appropriate storage systems, including relational, NoSQL and analytical databases
Apply multiple types of machine learning techniques to different use cases
Deploy machine learning models in production
Monitor data pipelines and machine learning models
Design scalable, resilient distributed data intensive applications
Migrate data warehouse from on-premises to Google Cloud
Evaluate and improve the quality of machine learning models
Grasp fundamental concepts in machine learning, such as backpropagation, feature engineering, overfitting and underfitting.

Course content

22 sections • 115 lectures • 6h 19m total length

Welcome2:12
Preparing for the Google Cloud Professional Data Engineer Exam3:27
Review the exam guide to understand the data engineer exam and take the assessment test, then build hands-on Google Cloud skills with $300 credit and study guides to prepare.

Introduction to Object Storage7:13
Understand what object storage is used for
Options for Loading Data3:45
Use gsutil, Transfer Service and other methods to upload data.
Access Controls for Cloud Storage2:15
Use IAM and access control lists to limit access to data in Cloud Storage
Lifecycle Policy Management2:26
Use policies to manage objects in Cloud Storage
Using Cloud Storage Console5:16
Use GCP console to manage Cloud Storage
Exercise: Cloud Storage0:13
Check your understanding of Cloud Storage
Solution: Cloud Storage1:39
Know the solution to the exercise.

Introduction to Relational Databases7:56
Effectively use GCP's managed relational database options
When to use Cloud SQL2:30
Use Cloud SQL for regional and zonal relational databases
Creating a Cloud SQL Database5:00
Create Cloud SQL databases
Monitoring Cloud SQL2:12
Learn to monitor Cloud SQL databases using the cloud console, track metrics like CPU, memory, storage, and ingress/egress, adjust time spans, and spot performance trends.
Introduction to Cloud SQL Auth Proxy3:47
Learn how to securely connect to Cloud SQL with the Cloud SQL Auth Proxy, Google's recommended solution that uses service account credentials and automatic TLS for public or private endpoints.
Installing Cloud SQL Auth Proxy4:33
Install the Cloud SQL Auth proxy on a Compute Engine VM by installing the Postgres client, downloading the proxy, and configuring it with your instance name to enable connections.
Exercise: Create a Cloud SQL Database0:31
Check your knowledge of Cloud SQL
Solution: Create a Cloud SQL Database2:32
Review the correct way to deploy a Cloud SQL database

Introduction to Cloud Firestore & Document Databases3:01
Entities and Kinds2:01
Learn how Google Cloud Datastore Firestore uses entities and kinds to model data with flexible properties, JSON-like key-value structures, and support for atomic values, arrays, and other entities.
Indexing in Cloud Firestore1:47
Understand how indexing controls queries in Cloud Firestore, with automatic single-field indexing for maps and arrays and manual composite indexes for multi-attribute filters.
Creating Entities3:55
Explore cloud data store basics by creating and viewing entities and kinds, using namespaces, and leveraging flexible properties across departments and products.
Querying Entities2:48
Creating Kinds and Namespaces2:26
Working with Transactions1:58
Cloud Firestore supports atomic transactions with serializable isolation, ensuring all operations succeed or fail together and reads see a consistent snapshot.
Exercise: Create a Kind and Entities0:27
Practice creating a Cloud Firestore kind named store and defining an entity with an auto-generated ID, a store name, and the city where the store is located.
Solution: Creating Kinds and Entities1:22
Create a kind named store and add entities with properties like name and city in the default namespace using numeric key identifiers, after completing initial setup and selecting a mode.

Introduction to Bigtable and Wide-Column Databases4:32
Creating a Bigtable Instance3:11
Designing Row-keys for Bigtable5:32
Design row keys in Bigtable to evenly distribute reads and writes across nodes and Colossus, avoid hotspots, and use high cardinality prefixes with time-reversed components.
Query Patterns and Denormalization4:01
Designing for Time Series Data with Bigtable8:05
Learn design patterns for time series data in Bigtable, including time buckets, row keys, and single event, serialized and unserialized storage strategies, with tradeoffs in performance and storage.
Check Your Knowledge: Creating a Time Series Database

Introduction to BigQuery and Analytical Databases5:18
Discover BigQuery, Google's serverless analytical data warehouse that uses SQL for analytics. Leverage data sets, tables, views, and materialized views with federated queries across storage systems.
BigQuery Scalar Datatypes1:09
BigQuery Nested and Repeated Fields1:28
Querying Scalars, Nested and Repeated Fields4:31
Explore BigQuery's sql interface with simple selects and unnest arrays and nested structures. See how to reference records, arrays, and BigQuery's analytical nature, not a relational database.
Exercise: Querying BigQuery Public Datasets0:27
Run a hands-on BigQuery query on the Census Bureau International and Midyear Population public dataset to list countries with a 2021 population over 100 million.
Solution: Querying BigQuery Public Datasets0:59
Query the census bureau international mid-year population table in BigQuery public data using standard SQL to list countries with a mid-year population over 100 million in 2021.
Access Controls in BigQuery6:25
Learn how to apply access controls in BigQuery with IAM at organization, project, dataset, and table levels, use column level security with Data Catalog tags, and understand BigQuery roles.
Partitioning Tables4:11
BigQuery partitions to boost query performance and reduce costs by scanning less data, using ingestion time, timestamp, null, and integer range partitions, with partition filters; partitioning is preferred over charting.
Clustering Partitioned Tables1:06
Loading Data into BigQuery3:36
Load data into BigQuery from Google Cloud storage, using streaming inserts, bulk loads, Avro or Parque formats, and Cloud Dataflow for transformations with manual, API, or bq cli loading.
BigQuery and IAM8:08
Learn how to control BigQuery access with IAM by assigning roles to identities, understanding permissions, and applying predefined, custom, and basic roles at organization, project, dataset, or table levels.
Column Level Security2:53
Define policies and taxonomy tags to control column level access to sensitive data, apply policy tags to columns, and enforce taxonomy-based access for highly critical information.
Row Level Security2:35

Assessing the Current State of a Data Warehouse3:19
Prepare and discover data warehouse workloads, assess the current state, plan the migration, execute offload or full migrations, and validate results in BigQuery.
Schema and Data Transfer3:06
Data Pipelines1:57
Data pipelines drive data warehousing by processing data through extraction, transformation, and load, enabling data enrichment and realtime analysis, then loading into the data warehouse.
Reporting and Analysis1:45
Explore reporting and analysis for data warehouse migrations, featuring descriptive, predictive, and prescriptive analytics, and leverage Data Studio, Looker, and BigQuery for OLAP querying and BigQuery Geo Viz.
Data Governance2:20
Check Your Knowledge: Data Warehouse Migration

Using Caching to Improve Performance4:28
Learn how Cloud Memory Store, a managed caching service for Redis and Memcached, delivers low-latency caching, high availability, and seamless integration with App Engine, Compute Engine, Cloud Functions, and Kubernetes.
Cloud Memorystore Data Structures2:02
Creating a Cloud Memorystore Cache4:01
Create a Redis cache in Cloud Memorystore, selecting an instance, region, and basic tier, set capacity and parameters, note costs, configure networking and IP/port, then delete the cache.
Check Your Knowledge: Redesign a Distributed Application for High Availability

Introduction to Cloud Composer2:52
learn how Cloud Composer, a managed Apache Airflow service, defines and runs DAG-based workflows as Python scripts stored in cloud storage, with logging via the Airflow web interface.
Cloud Composer Architecture3:03
Introduction to Cloud Data Fusion3:07
Discover cloud data fusion, a managed, code-free ETL tool built on CDAP that connects AWS, GCP, and Azure with 150-plus connectors via a drag-and-drop interface.

Requirements

Understanding of basic cloud computing concepts such as virtual machines and databases.
One year or more experience working with data management or data analysis

Description

The need for data engineers is constantly growing and certified data engineers are some of the top paid certified professionals. Data engineers have a wide range of skills including the ability to design systems to ingest large volumes of data, store data cost-effectively, and efficiently process and analyze data with tools ranging from reporting and visualization to machine learning. Earning a Google Cloud Professional Data Engineer certification demonstrates you have the knowledge and skills to build, tune, and monitor high performance data engineering systems.

This course is designed and developed by the author of the official Google Cloud Professional Data Engineer exam guide and a data architect with over 20 years of experience in databases, data architecture, and machine learning. This course combines lectures with quizzes and hands-on practical sessions to ensure you understand how to ingest data, create a data processing pipelines in Cloud Dataflow, deploy relational databases, design highly performant Bigtable, BigQuery, and Cloud Spanner databases, query Firestore databases, and create a Spark and Hadoop cluster using Cloud Dataproc.

The final portion of the course is dedicated to the most challenging part of the exam: machine learning. If you are not familiar with concepts like backpropagation, stochastic gradient descent, overfitting, underfitting, and feature engineering then you are not ready to take the exam. Fortunately, this course is designed for you. In this course we start from the beginning with machine learning, introducing basic concepts, like the difference between supervised and unsupervised learning. We’ll build on the basics to understand how to design, train, and evaluate machine learning models. In the process, we’ll explain essential concepts you will need to understand to pass the Professional Data Engineer exam. We'll also review Google Cloud machine learning services and infrastructure, such as BigQuery ML and tensor processing units.

The course includes a 50 question practice exam that will test your knowledge of data engineering concepts and help you identify areas you may need to study more.

By the end of this course, you will be ready to use Google Cloud Data Engineering services to design, deploy and monitor data pipelines, deploy advanced database systems, build data analysis platforms, and support production machine learning environments.

ARE YOU READY TO PASS THE EXAM? Join me and I'll show you how!

Who this course is for:

Cloud engineers and architects who want to pass the Professional Data Engineer exam
Data engineers who want to learn about Google's advanced tools and services for data engineering
Data scientists and data engineers who want to understand machine learning concepts
Cloud application developers who want to use machine learning to build applications
Devops engineers who need to support data engineering pipelines and machine learning models

Google Cloud Professional Data Engineer: Get Certified 2022

What you'll learn

Explore related topics

Course content

Introduction2 lectures • 6min

Cloud Storage for Data Engineering7 lectures • 23min

Relational Databases - Cloud SQL8 lectures • 29min

Relational Databases - Cloud Spanner3 lectures • 9min

NoSQL Databases: Cloud Firestore9 lectures • 20min

NoSQL Databases: Bigtable5 lectures • 25min

Analytical Databases: BigQuery Data Management13 lectures • 43min

Planning for Using a Data Warehouse5 lectures • 12min

Caching Data and Cloud Memorystore3 lectures • 11min

Workflows and ETL/ELT3 lectures • 9min

Requirements

Description

Who this course is for: