Understanding Regression Techniques

Explore multiple linear regression by adding a second independent variable, study hours, to predict GPA alongside attendance, and learn to visualize relationships with scatter plots.

Explore multiple linear regression and interpret coefficients as slopes for each independent variable, noting that attendance and study have positive effects on GPA with coefficients 0.71 and 0.59.

Compare models using the r squared value, showing that the multiple linear regression with attendance and study explains 90 percent of GPA variation, outperforming simple regression.

Learn how p values indicate statistical significance in multiple linear regression, with separate coefficients showing significance below 0.05 for attendance and study, supporting including both in the model.

Assess model fit and residuals by examining r-squared, significant coefficients, and predicted versus actual GPA using attendance and study in a linear regression, noting residuals can be positive or negative.

Encode binary, two-category variables like gender as zeros and ones to include them in multiple linear regression. Interpret the coefficient as the difference between categories relative to the reference group.

Learn how to encode categorical variables in regression using dummy variables, select a reference group, and interpret coefficients that compare each category to the referent.

Explore how quadratic terms capture nonlinear relationships in linear regression. Using an English GPA example, include English squared to improve fit, and assess significance with p-values and r-squared.

Assess the fit of a linear regression model by verifying assumptions after fitting and comparing predicted to observed GPA values with a plot near the y=x line.

Assess residuals in linear regression to verify normality, independence, and constant variance, using a histogram overlaid with a normal curve to visualize deviations from normality.

Assess the independence of residuals by plotting residuals against fitted values or independent variables to check for patterns. A random scatter supports independence, while a U shape pattern signals dependence.

Assess the constant variance by plotting residuals against x or fitted values. The example shows increasing residual spread from left to right, indicating the variance is not constant.

Identify multicollinearity by exploring correlations among independent variables and using the variance inflation factor (VIF). If a VIF exceeds 10, multicollinearity exists and you may consider removing the problematic variable.

Identify and diagnose outliers in regression analysis to understand their influence on model fit and slope, using diagnostics to decide how to handle them.

Identify influential observations in linear regression by removing each data point and re-fitting, using three statistics to measure result changes and flag outliers.

Examine variable selection in linear regression, including forward selection, backward elimination, and stepwise regression, and compare models using adjusted r squared, with theory and data guiding choices.

Analyze a university student dataset to see how GPA relates to English grade, college type, credits, gender, attendance, siblings, income, and work status using simple linear models.

Examine how to assess GPA with continuous predictors using scatter plots and simple or quadratic regression. Learn to interpret p-values and significance for attendance, English, income, and siblings.

Include binary variables in regression by coding college and gender as 0/1, interpreting coefficients as comparisons to the base groups, with significant GPA differences guiding future models.

Include the categorical work variable with multiple groups, using 'do not work' as the reference, and note that part-time GPA is 4.98 higher, full-time 2.61 lower, with p<0.05.

Explore building a multiple regression model with quadratic terms for attendance, income, and other predictors, interpret p-values, compare models with and without gender, and assess model assumptions and predictions.

Assess model fit in linear regression by r squared at 0.88, explaining 88 percent of GPA variation; compare predicted versus observed GPA on a scatter plot near y = x.

Check linear regression assumptions by examining residuals' normality and constant variance, using histograms, quantized normal plot, and tests like Shapiro-Wilk and Breusch-Pagan, with robust standard errors.

Assess multicollinearity in a multiple linear regression by examining VIF values, noting that income and income squared are correlated by design, while robust standard errors address heteroskedasticity.

Explore how diagnostic tools identify outliers and influential observations in multiple regression using added variable plots, revealing when a few points skew results.

Identify influential observations by calculating F statistics and Cook's D, and visualize with an added variable plot; compare full and reduced models to assess impact.

Visualize regression results with graphs to compare GPA across engineering and business students, and show how attendance, English grade, work status, and siblings influence GPA.

Use a two-by-two contingency table to compare binary outcomes—withdrawal versus finish—across engineering and business students, illustrating logistic regression implications.

Calculate odds from withdrawal probabilities using odds = p / (1 − p), compare engineering and business students, and interpret values greater than, less than, or equal to one.

Calculate the odds ratio by dividing engineering odds by business odds to compare withdrawal likelihoods, revealing engineering withdrawal is 4.5 times higher than for business students.

Explore how to compute probabilities and odds for a binary outcome in a two-by-three table, derive odds ratios between groups, and link these ideas to logistic regression.

Explore logistic regression with a single independent variable, linking odds of an event to x via e^(a x + b), and interpret the odds ratio as e^a.

Understand how logistic regression interprets the coefficient a as an odds multiplier via e^a, and interpret examples with heart condition risk by age and weekday outings by grade.

Learn how to code binary variables as 0 and 1 in regression, interpret logistic model coefficients, and compare group odds using odds ratios, with examples on gender and withdrawal.

Learn how binary and continuous predictors combine in a regression model with multiple variables, using college and courses as examples, and interpret coefficients as odds and odds ratios via e^coefficient.

Encode a three-level categorical variable with two binary indicators in logistic regression, comparing each group to the reference, and interpret coefficients as odds ratios for withdrawal.

Identify nonlinearity in linear and logistic regression by using a scatter plot to assess linearity, and apply a non-graphical box test with a variable times log(variable) to determine significance.

Explore graphical tests for nonlinearity in outcomes using the lowest graph, which smooths data, and apply the linearity of slopes test by categorizing a continuous variable in logistic regression.

Learn how linear regression predicts a dependent variable and how logistic regression predicts the probability of an event. Visualize predicted probabilities against GPA to reveal non-linear risk patterns.

Compare the fitted logistic regression model to a constant-only model using the likelihood ratio test to assess improvement; a p-value below 0.05 indicates a significantly better fit.

Explore the Hosmer-Lemeshow goodness-of-fit test for logistic regression, which groups data into about ten, compares observed versus fitted values, and uses p-values to judge model fit.

Explore how classification tables evaluate logistic regression fit by selecting a probability cut-off, balancing sensitivity and specificity, and using p values to classify observed outcomes.

Analyze goodness of fit with ROC analysis by measuring area under the curve. Interpret AUC values to gauge discrimination: 0.5 none, 0.17–0.8 acceptable, 0.18–0.9 excellent, 0.9 or above outstanding.

Explore residuals in logistic regression, including standardized, deviance, and delta x residuals, and learn to plot them against predicted probability to assess model fit and identify potential problems.

Assess influential observations in logistic regression by examining residuals and influence statistics, using a residuals versus predicted probabilities plot to decide if observations should be excluded.

Apply logistic regression to a university dataset describing student withdrawal. Analyze the binary outcome withdraw and predictors: college, gender, GPA, semester, and course level.

Examine how GPA, a continuous variable, relates to course withdrawal using linear and logistic regression, illustrated with scatter plots, averages, and odds ratios.

Assess linearity in linear and logistic regression using the box to dwell test with a GPA times Lin GPA term, and decide on a quadratic term based on p-values.

Use graphical tests to assess linearity, such as the lowest curve and slopes test on GPA in logistic regression, revealing nonlinearity and the need for a quadratic term in regression.

Explore binary variables like gender and college using logistic regression to estimate odds ratios and p-values, comparing withdrawal rates between engineering and business students.

Explore categorical variables in logistic regression by analyzing semester and course level, compare odds of withdrawal, collapse categories, and interpret p-values to simplify models.

Build a multiple logistic regression with GPA, GPA squared, spring, and Level 3; drop college. Interpret odds: male vs female 1.53, spring 1.22, 200 level 1.97, 500 level 0.07.

Assess logistic regression fit using likelihood ratio and Hosmer-Lemeshow tests, set optimal cut-off by balancing sensitivity and specificity, and confirm good fit with a 0.78 AUC and 71.72% correct classifications.

Identify outliers and influential observations by examining standardized residuals, device residuals, and delta x residuals plotted against the probability of withdrawal; reveal several outliers and discuss how to address them.

Identify influential observations and outliers using hat diagonal statistics and delta beta statistics, noting points above two times the average and the single case with delta beta greater than one.

Combine residuals and influence in a graph for logistic regression, where circle sizes reflect influence statistic. Identify influential outliers and compare models with and without them to decide on inclusion.

Visualize logistic regression by plotting predicted withdrawal probability against GPA, noting that GPA above 80 drives withdrawal to near zero and gender differences are strongest at 200-level courses.

Examine count and two-by-two tables to summarize failure data by business and engineering students, compute totals, and compare failure frequencies while accounting for total courses.

Calculate and compare the risk of course failure for business and engineering students using failure counts over total courses, revealing higher risk for engineering students.

Compare risks of course failure between business and engineering students using the incidence rate ratio; a value of 0.878 means business risk is 0.878 times the engineering risk.

Analyze failure risk across business, engineering, and life sciences with two-by-three tables. Compute incidence rate ratios to compare each group's risk against others.