Theory of Gaussian Process Regression for Machine Learning

Name: Theory of Gaussian Process Regression for Machine Learning
Rating: 4.2 (133 reviews)

Introduction to a probabilistic modelling tool for Bayesian machine learning, with application in Python

Created byFoster Lubbe

Last updated 8/2021

English

What you'll learn

The mathematics behind an algorithm such as the scikit-learn GaussianProcessRegressor algorithm
The benefits of Gaussian process regression
Examples of Gaussian process regression in action
The most important kernels needed for Gaussian process regression
How to apply Gaussian process regression in Python using scikit-learn

Course content

3 sections • 14 lectures • 1h 8m total length

Introduction3:11
Explore Gaussian process regression as a non-parametric Bayesian framework for interpolation and prediction, using kernels to capture function properties and quantify uncertainty with minimal prior assumptions.
Reading 1
The conditional of a Gaussian8:25
Learn how the conditional of a Gaussian underpins Gaussian process regression by deriving the mean and covariance of x1 given x2 with a linear-algebra recipe (Theorem 1).
Reading 2
An example of finding the conditional3:21
Apply Theorem 1 to a joint Gaussian (x1, x2) with 0.95 correlation to find x1 given x2 = -1, producing a Gaussian with mean -0.95 and a covariance of 0.0975.
Reading 3
Supervised learning with the Gaussian process9:11
Define a Gaussian process over functions and condition on training inputs and outputs to obtain the predictive distribution, mean, covariance, and kernel-based interpolation for new inputs in supervised learning.
Reading 4
An example of supervised learning3:55
Demonstrate a hypothetical Gaussian process regression for supervised learning with a zero-mean prior and kernel-based covariance, refined by training data into a 95% confidence band.
Reading 5

Kernels and their usefulness4:33
Discover how gaussian process regression uses kernels such as linear, radial basis, rational quadratic, and periodic to encode similarities, with hyperparameters guiding priors and posteriors.
Reading 6
Combining Kernels7:30
Structure Discovery in Nonparametric Regression through Compositional Kernel Search: https://arxiv.org/pdf/1302.4922.pdf
A Practical Guide to Gaussian Process Regression for Energy Measurement and Verification within the Bayesian Framework: https://www.mdpi.com/1996-1073/11/4/935
The Kernel Cookbook: https://www.cs.toronto.edu/~duvenaud/cookbook/
Classic Gaussian process regression examples4:00
Explore classic Gaussian process regression with real data examples, modeling seasonal variation, long-term trends, and irregularities using kernels like exponential sine squared, radial basis function, rational quadratic, and white noise.
Reading 7
Scikit-learn3:50
Explore scikit-learn, a versatile Python machine learning library for supervised and unsupervised tasks, offering classification, regression, clustering, and preprocessing with comprehensive documentation and a BSD-licensed open source.
Scikit-learn GaussianProcessRegressor3:28
Applying Gaussian process regression to real-world data I4:48
Apply Gaussian process regression to real world weather data, building a kernel-based model to interpolate global horizontal irradiance using time inputs and various weather streams, exploring multiple kernels.
Applying Gaussian process regression to real-world data II3:33
Apply gaussian process regression to real-world data using kernels like constant, exponential sine squared, and rational quadratic; train with restarts, fit to X, Y, and predict with confidence bounds.
Applying Gaussian process regression to real-world data III2:53
Wrap: summarizing the process4:30
Explore gaussian process regression, detailing training data, prior and posterior kernels, hyperparameter optimization, and interpolation and prediction with confidence bounds.

Requirements

A basic understanding of linear algebra
Basic experience with coding

Description

Probabilistic modelling, which falls under the Bayesian paradigm, is gaining popularity world-wide. Its powerful capabilities, such as giving a reliable estimation of its own uncertainty, makes Gaussian process regression a must-have skill for any data scientist. Gaussian process regression is especially powerful when applied in the fields of data science, financial analysis, engineering and geostatistics.

This course covers the fundamental mathematical concepts needed by the modern data scientist to confidently apply Gaussian process regression. The course also covers the implementation of Gaussian process regression in Python.

Who this course is for:

Data scientists, engineers and financial analysts looking to up their data analysis game
Anybody interested in probabilistic modelling and Bayesian statistics

Theory of Gaussian Process Regression for Machine Learning

What you'll learn

Explore related topics

Course content

Fundamentals5 lectures • 28min

Application9 lectures • 39min

Assignment0

Requirements

Description

Who this course is for: