Udemy
Turn what you know into an opportunity and reach millions around the world.
Learn More
Your cart is empty.
Keep shopping
ETL using Python: from MySQL to BigQuery
Rating: 4.4 out of 5(936 ratings)
6,132 students

ETL using Python: from MySQL to BigQuery

A course for supercharged analysts
Created byOscar Valles
Last updated 4/2026
English
German [Auto],English [Auto],

What you'll learn

  • Connect to MySQL using Python
  • Connect to BigQuery using Python
  • ETL data from MySQL to BigQuery using Python
  • Setting up their environment to use Python with MySQL and BigQuery

Course content

5 sections • 27 lectures • 2h 58m total length

  • Introduction5:07

    Explore ETL with Python to move data from MySQL to BigQuery, detailing extract, transform, load steps, tool options, and practical setup for analysts and engineers.

  • Installing Python3:17

    Install Python across Linux, Mac, and Windows by checking with python -V, using python.org downloads or Homebrew, and running brew install python to obtain the latest version.

  • Virtual Environments5:32

    Learn to set up and manage python virtual environments across Windows, macOS, and Linux, using Anaconda on Windows and Real Python guides to install Python versions with activation and deactivation.

  • Creating a Google Account3:37

    Set up a Google Cloud account to explore Google Cloud Platform services, including BigQuery, while understanding the free $300 credits and the 100 TB monthly query allowance.

  • BigQuery Project, Dataset and Tables6:20

    Navigate the Google Cloud console to set up a BigQuery project, create a dataset and tables, preview data, and run a sample query filtering year 2011.

  • Installing the Google SDK7:58

    Learn to install the Google Cloud SDK to enable Python-based ETL workflows from MySQL to BigQuery, including downloading, configuring your environment, and initializing with gcloud init.

  • Google Authentication1:52

    Authenticate your Python ETL to Google Cloud by running gcloud auth application-default login, granting access to your Google account, and using the saved credential files to connect to BigQuery.

  • Storing Connection Properties3:15

    Please note, connection properties have been updated:

    host: 82.197.82.63

    username: u479841347_user

    password: LearnSQL123

    database: u479841347_sql_course

    port: 3306

  • Installing Needed Modules7:02

    Set up a Python ETL environment by creating and using a virtual environment, then install pandas, numpy, mysql-connector, Google Cloud, and PyArrow for MySQL to BigQuery workflows.

  • ETL Overview4:06

    Explore the etl process: extract data from MySQL, transform it with sql or Python (Pandas), and load the transformed dataset into BigQuery, with notes on elt and when transformations occur.

Requirements

  • Python installed (e.g. virtual environment, anaconda, etc...)
  • Familiarity with SQL
  • Familiarity with Python
  • GCP Account for BigQuery Access
  • An IDE like VS Code or PyCharm

Description

This is a direct and to the point course that will get you quickly ETL'ing data from MySQL to BigQuery.

The lessons in this course are broken out into short How-Tos. Therefore you can take this course over the weekend and be ready to show off your skills on Monday morning!

Things that we will cover:

  • Setup

    • Setting up a GCP Account

    • Credential and Authentication for security

    • Python Environment Setup

  • Extract

    • Use Python to connect to MySQL

    • Use Python's pandas to export data

    • Python library usage for saving files to file paths

  • Transform

    • Use Python functions to transform data

    • Use Python pandas to transform data

    • Use inline SQL during Extract for data transformation

  • Load

    • Use the BigQuery Python library

    • Connect to BigQuery

    • Load data to BigQuery

    • Incremental Loads vs Truncate and Load

    • Other data handling options during Load

After taking this course, you'll be comfortable with the following pretty cool things:

  • Connect to MySQL using Python

  • Learn how to obscure your database credentials so you're not exposing them in your code

  • Usage of the os module for the purpose of saving files and hard coding fewer things.

  • Use both Python and the pandas library to transform data on the fly during the Transformation phase of your ETL

  • Learn how to use GBQ's modules/libraries to make the loading of the data a very easy, straightforward task

Have fun, enjoy and keep growing!

Who this course is for:

  • Business Intelligence Analysts
  • Data Analysts
  • Beginner Data Engineers
  • Beginner Software Developers
  • Data Power Users

Report abuse