Linear Regression, GLMs and GAMs with R

Name: Linear Regression, GLMs and GAMs with R
Rating: 4.3 (266 reviews)

How to extend linear regression to specify and estimate generalized linear models and additive models.

Created byGeoffrey Hubona, Ph.D.

Last updated 9/2020

English

English [Auto],

What you'll learn

Understand the assumptions of ordinary least squares (OLS) linear regression.
Specify, estimate and interpret linear (regression) models using R.
Understand how the assumptions of OLS regression are modified (relaxed) in order to specify, estimate and interpret generalized linear models (GLMs).
Specify, estimate and interpret GLMs using R.
Understand the mechanics and limitations of specifying, estimating and interpreting generalized additive models (GAMs).

Course content

5 sections • 69 lectures • 7h 54m total length

Introduction to Course1:51
Explore linear regression and its extensions with generalized linear models and generalized additive models, relaxing linearity assumptions to model binary and count data with nonlinear polynomials and smooths.
Preliminaries: Installing R, RStudio, R Commander, Course Materials and Exercise5:16
Install the R software, RStudio, and the R Commander, download the materials zip from view resources, and complete the exercises while using the discussion board for questions.
Beginning Agenda (slides)8:18
Explore linear models, generalized linear models, and generalized additive models in R, with a focus on data types—count, proportion, and binary—and how to choose appropriate distributions.
What is Linear Modeling? (slides, part 1)5:11
The term "linear" refers to the fact that we are fitting a line. The term model refers to the equation that summarizes the line that we fit. The term "linear model" is often taken as synonymous with linear regression model.
Assumptions of Linear Modeling (slides, part 2)6:08
Assumptions of Linear Models (regression):

The residuals are independent

The residuals are normally distributed

The residuals have a mean of 0 at all values of X

The residuals have constant variance
Desirable Properties of Beta-hat (slides, part 3)7:19
Explain that beta-hat is a random normal estimator with mean equal to the true beta, discuss its sampling variance, and reveal an unbiased sigma-squared estimator from residuals for confidence intervals.
Example: Estimate Age of Universe (slides)4:39
Estimate the age of the universe by fitting a linear model to galaxy velocities and distances using Doppler measurements and Hubble's constant, testing the Big Bang theory.
Example: Estimate Age of Universe Live in R (part 1)7:44
Explore live R setup using the R commander to load packages and run Hubble data analyses. Use the lm function to perform linear regression on velocity and distance.
Example: Estimate Age of Universe Live in R (part 2)9:22
Learn to fit a simple linear regression in R, including suppressing the intercept and estimating the beta coefficient, then use summary and residual plots to assess model fit.
Example: Estimating Age of the Universe (part 3)8:50
Finish Example and More Notes on Linear Modeling8:31
Explore linear modeling and generalized linear and additive models in R, compare models with anova and aic, and interpret confidence intervals, t-statistics, and residual diagnostics.
Linear Modeling Exercises1:48

Introduction to GLMs (slides, part 1)6:58
In statistics, the generalized linear model (GLM) is a flexible generalization of ordinary linear regression that allows for response variables that have error distribution models other than a normal distribution. The GLM generalizes linear regression by allowing the linear model to be related to the response variable via a link function and by allowing the magnitude of the variance of each measurement to be a function of its predicted value.
Introduction to GLMs (slides, part 2)7:29
Explore generalized linear models, distinguishing the linear predictor from the transformed predictive value via link functions, including canonical links, and interpret coefficients through the inverse of the link function.
Introduction to GLMs (slides, part 3)7:50
Introduction to GLMs (slides, part 4)6:44
Explain probability vs. odds and the logit link in binomial GLMs. Demonstrate predicting heart attack probability from CK levels and loading data in R with read.csv.
Example: Binomial (Proportion) Model with Heart Disease (part 1)7:50
Proportion data has values that fall between zero and one. Naturally, it would be nice to have the predicted values also fall between zero and one. One way to accomplish this is to use a generalized linear model (glm) with a logit link and the binomial family.
Example: Binomial (Proportion) Model with Heart Disease (part 2)7:26
Model binomial heart attack data with a logit GLM, from proportion to two-column counts, and interpret the sigmoid relationship with CK levels.
Example: Binomial (Proportion) Model with Heart Disease (part 3)8:16
Example: Binomial (Proportion) Model with Heart Disease (part 4)6:22
GLM Exercises1:05
Explore logistic regression on binary data, interpret results, and extend with Poisson and quadratic GLMs and GAMs in R, comparing model fit.

Current Agenda1:46
Linear Regression Exercise Solutions (part 1)7:31
Linear Regression Exercise Solutions (part 2)7:29
GLM Exercise Solutions (part 3)9:30
Explore glm exercise solutions with binomial family and logit link, modeling inhibition counts as a proportion of successes against log concentration. Assess linear and quadratic time models via anova-style tests.
Example: Poisson Model with Count Data (part 1)8:15
In statistics, Poisson regression is a form of regression analysis used to model count data and contingency tables. Poisson regression assumes the response variable Y has a Poisson distribution, and assumes the logarithm of its expected value can be modeled by a linear combination of unknown parameters. A Poisson regression model is sometimes known as a log-linear model, especially when used to model contingency tables.

Poisson regression models are generalized linear models with the logarithm as the (canonical) link function, and the Poisson distribution function as the assumed probability distribution of the response.
Example: Poisson Model with Count Data (part 2)9:29
Example: Binary Response Variable (part 1)4:43
Example: Binary Response Variable (part 2)6:12
Exercise: GLM to GAM1:40
Fit generalized additive models with smooth terms for area and isolation. Compare models with both terms to isolation only and analyze deviance to assess variable effects.
Example: Log-Linear Model for Categorical Data5:55
Log-linear analysis is a technique used in statistics to examine the relationship between more than two categorical variables.
More on Deviance and Overdispersion (slides)3:11

What are GAMS? (Crawley, slides, part 1)7:41
In statistics, a generalized additive model (GAM) is a generalized linear model in which the linear predictor depends linearly on unknown smooth functions of some predictor variables, and interest focuses on inference about these smooth functions. GAMs were originally developed by Trevor Hastie and Robert Tibshirani to blend properties of generalized linear models with additive models.
What are GAMs? (Crawley, slides, part 2)6:02
Demonstrate GAM Ozone Data (part 1)9:40
Explore generalized additive models for ozone data by fitting smooth, non-linear terms for radiation, temperature, and wind, and compare models with and without radiation using DCV and significance.
Demonstrate GAM Ozone Data (part 2)9:42
Compare generalized additive models for ozone data, test interactions between wind and temperature, and assess model fit with deviance, cross-validation, and smoothing splines to balance bias and variance.
General Approaches for Fitting GAMs (slides)2:44
Explore general approaches for fitting GAMs using penalized likelihood methods to control curvature and prevent overfitting, balancing multiple predictor shapes in the linear predictor.
What are GAMs? (Wood, slides, part 1)11:34
Univariate Polynomial GAMs (Wood, slides, part 2)7:27
Select a polynomial basis to build a univariate additive model and express f(x) as a fourth-order polynomial with estimated coefficients, scaling x to 0-1.
Univariate Polynomial GAMs (Wood, slides, part 3)5:52
Model y as an additive sum of univariate polynomial basis functions in GAMs. See how coefficients shape quadratic and higher-order terms, revealing how polynomial curves exhibit inflection points.
GAMs as 4th Order Polynomials (slides, part 1)6:21
Represent a univariate relation as a linear model using a fourth-order polynomial basis for GAMs, estimate coefficients by least squares on an X matrix, and form f(x) from these terms.
GAMs as 4th Order Polynomials (slides, part 2)4:29
GAMs as Regression Splines (slides)3:38
Examine regression splines built from cubic spline segments joined at knots to form a continuous curve, constraining the additive model and aiding estimation of the response.
Cubic Splines (slides, part 1)8:45
Identify knot locations for cubic splines using evenly spaced x-values or observed y-values, and construct the spline basis within a least-squares linear model to estimate y.
Cubic Splines (slides, part 2)4:21
Function to Establish Basis for Spline (slides)7:33
Build-a-GAM (slides, part 1)7:46
Build a GAM by constructing a basis function, generating a model matrix with knots, and fitting a linear model to predict y from scaled x; explore prediction, coefficients, and plotting.
Build-a-GAM (slides, part 2)10:16
Learn to build a GAM using regression splines, apply a lambda-based smoothing penalty on the second derivative, and compare knot spacing and model complexity for better predictions.
Build-a-GAM (slides, part 3)6:17
Builds a penalized regression spline with a second-derivative penalty matrix, forms an augmented model matrix with lambda, fits the model, and compares three lambda values to find the optimal fit.
Build-a-GAM Demonstration in R Script11:34
Demonstrate building a univariate GAM in R by scaling x to 0–1, constructing a spline model matrix with knots, and using lambda to control smoothing.
Build-a-GAM Cross Validation8:13
Explore generalized cross-validation to select the smoothing parameter lambda for GAMs, using leave-one-out cross-validation and GCV to minimize prediction error and fit the optimal model.
Bivariate GAMs with 2 Explanatory Independent Variables (slides, part 1)9:17
This lecture extends the univariate additive model to a two-term additive model with two predictors. It addresses identifiability and uses penalized regression splines with cross-validation.
Bivariate GAMs with 2 Explanatory Independent Variables (slides, part 2)7:31
Exercises1:33
Practice generalized additive modeling by simulating data, fitting fifth- and tenth-order polynomials, and comparing cubic regression spline fits with predict to see how polynomial order and basis affect the lines.

Current Agenda (slides)5:23
Explore generalized linear models and generalized additive models using real-world examples, from brain imaging to air pollution, and compare models with isotropic and tensor product smooths.
Cherry Trees and Finer Control (slides, part 1)8:10
Finer Control of GAM (slides, part 2)10:52
The lecture compares smoothing options for gams, from thin plate regression splines to penalized cubic regression splines, and shows how basis dimension and gamma affect degrees of freedom and gcv.
Using Smoothers with More than One Predictor (slides)7:04
Explore smooth terms with multiple predictors in generalized additive models, modeling height and girth with isotropic and tensor product smooths for scale-insensitive, stable estimates.
More on Alternative Smoothing Bases (slides)8:06
Compare thin plate and cubic regression splines, then show tensor product smoothing improves stability and scale handling for multi covariate GAMs.
Parametric Model Terms (slides)8:29
Example: Brain Imaging (part 1)7:51
Predict brain activity levels from x and y voxel locations using smooth models in a generalized additive model framework, with region effects and checks for model selection and fit.
Example: Brain Imaging (part 2)8:09
Example: Brain Imaging (part 3)7:38
Example: Brain Imaging (part 4)7:03
Learn how generalized additive models predict brain activity using isotropic smooth terms, additive terms, comparing isotropic interaction (M2) with an additive model (M3). Assess model fit via deviance and prediction.
Example: Brain Imaging (part 5)7:41
Assess brain imaging models by comparing additive, isotropic interaction, and tensor product fits using analysis of deviance and DCV; reject additive in favor of isotropic or tensor product.
Example: Air Pollution in Chicago (part 1)9:33
Explore a semi-parametric generalized additive model for Chicago air pollution and death rates. Smooth time and covariates, use log response for Poisson data, and diagnose with cam check residual plots.
Example: Air Pollution in Chicago (part 2)9:17
Air Pollution in Chicago (part 3)4:40
More Exercises5:41

Requirements

Students will need to install R and R Commander software but ample instruction for doing so is provided.

Description

Linear Regression, GLMs and GAMs with R demonstrates how to use R to extend the basic assumptions and constraints of linear regression to specify, model, and interpret the results of generalized linear (GLMs) and generalized additive (GAMs) models. The course demonstrates the estimation of GLMs and GAMs by working through a series of practical examples from the book Generalized Additive Models: An Introduction with R by Simon N. Wood (Chapman & Hall/CRC Texts in Statistical Science, 2006). Linear statistical models have a univariate response modeled as a linear function of predictor variables and a zero mean random error term. The assumption of linearity is a critical (and limiting) characteristic. Generalized linear models (GLMs) relax this assumption of linearity. They permit the expected value of the response variable to be a smoothed (e.g. non-linear) monotonic function of the linear predictors. GLMs also relax the assumption that the response variable is normally distributed by allowing for many distributions (e.g. normal, poisson, binomial, log-linear, etc.). Generalized additive models (GAMs) are extensions of GLMs. GAMs allow for the estimation of regression coefficients that take the form of non-parametric smoothers. Nonparametric smoothers like lowess (locally weighted scatterplot smoothing) fit a smooth curve to data using localized subsets of the data. This course provides an overview of modeling GLMs and GAMs using R. GLMs, and especially GAMs, have evolved into standard statistical methodologies of considerable flexibility. The course addresses recent approaches to modeling, estimating and interpreting GAMs. The focus of the course is on modeling and interpreting GLMs and especially GAMs with R. Use of the freely available R software illustrates the practicalities of linear, generalized linear, and generalized additive models.

Who this course is for:

This course would be useful for anyone involved with linear modeling estimation, including graduate students and/or working professionals in quantitative modeling and data analysis.
The focus, and majority of content, of this course is on generalized additive modeling. Anyone who wishes to learn how to specify, estimate and interpret GAMs would especially benefit from this course.

Linear Regression, GLMs and GAMs with R

What you'll learn

Explore related topics

Course content

Introduction to Course and to Linear Modeling12 lectures • 1hr 15min

Generalized Linear Models (GLMs) Part 19 lectures • 1hr

Generalized Linear Models Part 211 lectures • 1hr 6min

Generalized Additive Models Explained22 lectures • 2hr 38min

Detailed GAM Examples15 lectures • 1hr 56min

Requirements

Description

Who this course is for: