
Set up a Google Cloud account with free credits to explore cloud storage, BigQuery, Pub/Sub, Cloud Functions, Vertex II, and free tier products.
Open the course console overview tab to search questions, post a new question about the current or all lectures, and receive responses from instructors and peers; connect via LinkedIn.
Learn how relational databases use tables, keys, indexes, views, and normalization to support structured data with strong consistency and transactional integrity. Compare when to choose relational, analytical, or non-relational options.
Choose Cloud SQL for a regional relational data store that supports asset transactions, PostgreSQL, MySQL, and SQL Server, offering high availability across zones and strong consistency.
Create a PostgreSQL instance in Cloud SQL via the cloud console, choose version 14 and a development configuration with private IP, automatic backups, and point-in-time recovery.
Learn how to add and delete users in Cloud SQL using built-in authentication or Cloud IAM, assign privileges, set password policies, and control host access for a MySQL instance.
Clone a Cloud SQL database for a quick development copy from current state or a point in time. Preserve the same configuration while the clone uses a new IP address.
Learn how to delete a cloud sql instance by disabling accidental deletion protection in the cloud console, editing the instance, saving changes, and confirming deletion by typing the instance name.
Use the Google Cloud pricing calculator to estimate Cloud SQL costs for MySQL, PostgreSQL, or SQL Server, comparing development and production configurations across region, storage, backups, and committed usage.
Harness the Cloud SQL auth proxy to securely connect to Cloud SQL via a secure tcp tunnel, using service account credentials and optional credential files for public or private endpoints.
Install the Cloud SQL auth proxy on a Compute Engine VM, using a lightweight micro instance, then install the PostgreSQL client and run the proxy with your instance name.
Enable SSL enforcement for a Cloud SQL instance with a public IP, then create and download the client and server certificates, and store keys securely in a safe location.
Export and import cloud sql databases using sql or csv formats, leveraging a cloud storage bucket, with options to offload to a temporary instance to reduce load.
Apply best practices for importing and exporting data in Cloud SQL: export from a read replica, use serverless exports, batch operations, and schedule via Cloud Scheduler, Pub/Sub, and Cloud Storage.
Explore Cloud SQL replication options, including read replicas in the same region, cross-region replicas, cascading replicas, and external read replicas, to offload work, reduce latency, and enable disaster recovery.
Create a read replica in Cloud SQL to distribute read workloads from the primary, enabling queries on a separate IP and using IAM authentication.
Create and manage Cloud SQL backups in the Cloud SQL console, choosing single region or multi-region locations for protection, durability, high availability, and learn how to restore or delete backups.
Enable automatic operating system and database updates in Cloud SQL as a managed service. Schedule maintenance windows and define update order to minimize connectivity loss and avoid peak periods.
Learn to use the database migration service to perform one-time copies or continuous migrations between Cloud SQL for MySQL instances, using private IP, VPC peering, and connection profiles.
Learn to manage cloud sql from the command line using gcloud, performing backup, connect, export, and import, plus instance operations like create, delete, clone, failover, and patch.
Explore alloy db, a fully managed postgres-compatible database with decoupled compute and storage, AI-driven optimization, and support for row and column storage.
Learn to create an AlloyDB cluster with high availability and read pools, configure a primary and a read pool, select region and SQL 14 compatibility, and prepare for connection.
Estimate AlloyDB costs using location, vCPU, memory, storage, backups, and high availability; development costs are a few hundred dollars, production clusters can reach tens of thousands monthly.
Cloud Spanner is a globally distributed relational database for structured data with a predefined schema and strong consistency. It's a managed service with automatic replication.
Create a Cloud Spanner instance by selecting regional or multi-regional configurations in us west one, choose processing units, review replicas, sla, and cost, and follow capacity guidance and database limits.
Create a Spanner database within an instance, then define a customer table with id, first name, last name, and birth date, using Google Standard SQL, DDL templates, and encryption options.
Add users to a Cloud Spanner instance and assign roles like Cloud Spanner admin or database reader. Review the resulting service accounts and owners that have access.
Delete a Cloud Spanner instance using the Cloud Console or gcloud commands, type the instance name Spanner instance one, and start the deletion process.
Leverage Cloud Spanner change streams to capture near real-time changes from the entire database or specific tables or columns, enabling analytics with BigQuery and event-driven logic via Pub/Sub.
Export data from Cloud Spanner to a Cloud Storage bucket via the console, choosing the bucket and region, and running a Cloud Dataflow job.
Learn how to export and import data to Cloud Spanner, including selecting source, configuring a transfer bucket, choosing a database dialect, naming the import job, and executing via Cloud Dataflow.
Explore gcloud spanner commands for backups, databases, instances, and operations, including create, delete, describe, and restore actions; manage IAM policy bindings, execute SQL, and update metadata for Cloud Spanner.
Learn to optimize cloud spanner performance by avoiding hot spotting through hashed keys, high-cardinality primary keys, and key distribution, and improve join efficiency with interleave tables for parent-child data.
Estimate Cloud Spanner costs with the Google pricing calculator by configuring regional or multi-regional deployments, compute units or nodes, storage, backups, and one-year commitment.
BigQuery is a serverless analytics data warehouse that scales to petabytes, uses SQL on a columnar, nested data store, and supports BigQuery ML, BI engine, and federated data access.
BigQuery, a highly managed analytics service, shifts focus from servers to data sets, schemas, and nested structs for analytics and data organization.
Load data into BigQuery from cloud storage, other cloud services, or local machines using batch or streaming methods; leverage UI, API, or bq cli with Dataflow transforms and schema auto-detect.
Explore partitioning and clustering in BigQuery to boost performance and lower costs, using ingestion time with underscore partition time, date partitions, null and unpartitioned cases, and partition filters.
Explore multi-level access controls in BigQuery, from organization and dataset to table and column levels, using IAM roles and policy tags defined in data catalog.
Estimate BigQuery costs by comparing on demand versus flat rate pricing, considering active and long-term storage, streaming inserts and reads, and total data scanned.
Explore Cloud Firestore, a managed NoSQL document database using a document model of key-value pairs and nested structures, with native and data store modes for semi-structured data.
Create and manage a Firestore collection and documents using a no sql document model with flexible schemas, storing attributes like product name, memory, disk size, color, and wheels.
Explore how Firestore uses indexes to locate documents efficiently, including automatic single-field indexes and customizable composite indexes, with exemptions at collection or collection group scope to optimize queries.
Export and import data in Cloud Firestore by selecting the entire database or specific collections, choosing the correct Cloud Storage bucket and region, and resolving regional errors to complete transfers.
Estimate Firestore costs by counting document reads, writes, deletes, and data stored in a serverless Firebase environment; understand how pricing scales from the free tier to high-volume usage.
Explore Bigtable, a wide column NoSQL database built on a sparse multi-dimensional array. It supports petabyte-scale data with low-latency writes and key-based reads for analytics.
Create a Bigtable table within your instance, using flexible, no-SQL schema and optional column families, and configure garbage collection policies to manage storage.
Create a table in big table by naming it, with optional column families to group related columns, and manage storage with a simple garbage collection policy.
Add a cluster to a Bigtable instance to enable high availability and distribute workload. Edit instance, pick a region such as us west 1 or us central 1, and save.
Design Bigtable row keys to balance performance and data location, avoiding linearly incrementing keys and low-cardinality prefixes, and starting with high-cardinality attributes like sensor IDs.
Learn denormalization patterns in Bigtable to improve read performance by storing related data in adjacent rows, leveraging the row key, and using column families for related attributes.
Use the Bigtable key visualizer to analyze usage patterns with a heat map, troubleshoot performance issues, and tune row key design to avoid hot spotting and improve table performance.
Learn how to delete a Bigtable instance to avoid unnecessary costs, including removing the instance, its clusters, and tables from the console.
Configure Bigtable with application profiles to route requests to single or multiple clusters, balancing read/write consistency, failover handling, and high availability based on your app's needs.
Estimate the cost of using big table with the Google Cloud pricing calculator and explore how cluster size, storage type, and run time affect monthly expenses.
Learn to roughly estimate a database size using a simplified customer and address model, factoring column sizes, row counts, index and overhead to calculate total bytes.
Estimate relational database size by computing average column size, average row size, and expected row count; include indexing and overhead, with platform-specific guidance from MySQL, PostgreSQL, and SQL Server.
Provision IOPS for Cloud SQL by evaluating IO performance requirements and calculating max IOPS from baseline and per-gig IOPS times disk size, considering regional vs zonal disks and VCPU count.
Monitor databases with Google Cloud Monitoring, using predefined and custom metrics, dashboards, alerting, uptime checks, cloud logging, log explorer, and routing via Cloud Router, with Pub/Sub and BigQuery for analysis.
Prefer managed databases to leverage the shared security model and automated patching; for self-managed setups, apply VPC firewall rules, dedicated servers, per-application service accounts, encrypted connections, and least-privilege access.
Use Datastream to capture changes and replicate data in near real time from Oracle, MySQL, and PostgreSQL to Cloud Spanner, Cloud SQL, BigQuery, and Cloud Storage.
Explore how Private Service Connect enables private, serverless networking across VPCs, letting you access Google APIs and services in other VPCs or from a SaaS provider via private endpoints.
Jan 2023: Newly updated with additional material on database migration, meeting performance requirements, Oracle migrations, and more.
Databases are the foundation of many of the applications and services that we use every day. Keeping them up and running is the responsibility of database engineers. In this course, you will learn essential skills and hear hard-learned lessons about designing, deploying, securing, maintaining and migrating highly scalable and highly available databases.
By the end of the course, you'll be ready to take the Google Cloud Professional Database Engineer Exam and take on the responsibilities of a database engineer working in Google Cloud.
The course begins with an introduction to relational databases and working with Cloud SQL, a regional, managed database service provided by Google Cloud. You will learn:
When to use Cloud SQL, Cloud Spanner, Cloud Firestore, Bigtable, BigQuery, and AlloyDB
How to create database instance in managed database services
Managing users
Planning for high availability and reliability
Database security best practices
Creating, managing, and cloning databases
Connecting securely to your databases
Monitoring, logging, and alerting
Best practices for importing, exporting, and backing up your databases
Understanding data migration processes and how to execute database migrations
Using specialized services, like Datastream, Database Migration Service and Oracle Bare Metal
Understanding how to estimate database sizes and configure IOPS to meet performance requirements
We'll also cover database maintenance and how to migrate a self-managed database to Cloud SQL. But we don't stop with Cloud SQL. Google Cloud offers AlloyDB, a scalable PostreSQL implementation that is designed to support both online transaction processing and analytical workloads. You will also learn about Cloud Spanner, the managed, globally scalable relational database offered by Google Cloud.
In addition to relational databases, we dig into the details of BigQuery, a petabyte scale data warehouse, and analytics service. You'll learn how to use BigQuery's console to navigate datasets, organize data using partitioning and clustering, and control access to data in BigQuery.
Bigtable and Cloud Firestore are two managed NoSQL databases available in Google Cloud. In this course, you'll see how to create Bigtable databases, add clusters to a Bigtable instance, and design multidimensional schemas that are highly performant in Bigtable. You'll also learn about Cloud Firestore, a document database, that provides flexible schemas while also offering indexing and a query language that makes it an ideal choice for a database backend for many services.
In addition to knowing about different kinds of databases and when to use them, database engineers have to maintain and troubleshoot databases. This course will help you learn how to estimate the size of databases, monitor databases as well as configure security and networking for databases. You'll also learn about essential data management practices including change data capture (CDC) using Datastream and database migration using Database Migration Service.
Database migrations are an essential step to deploying applications and services in the cloud and they are some of the most challenging operations to plan and execute. This course includes in-depth lectures on the characteristics of database migrations, database migration processes, executing database migrations, ensuring data consistency between source and target databases, using change data capture and differential querying, draining and switchover, as well as fallback planning.
Database engineers are in demand because they understand how to build, deploy, and manage critical information infrastructure.
Google Cloud has developed the Professional Cloud Database Engineer certification to distinguish database engineers with the necessary skills to operate high-performance databases in production environments in Google Cloud. Join us in this course to advance your career as a certified Professional Database Engineer.