Machine Learning 101 with Scikit-learn and StatsModels

Install and configure the Python data science environment using Anaconda, Python 3, and Jupyter Notebook, then install the relevant packages and learn the coding environment for machine learning.

Explore why Python and Jupyter power machine learning workflows with open source tools, cross-platform support, and browser-based notebooks.

Install the Anaconda distribution to get Python, the Jupyter Notebook, and data science packages, with a guided setup for Windows (also applicable to Mac and Linux).

Explore the Jupyter dashboard to manage notebooks, files, and folders, including uploading, renaming, moving, and deleting items, and run code in an interactive shell.

Explore the Jupyter dashboard, learn to create and run code cells with ctrl+enter or shift+enter, convert between code and markdown, and navigate with keyboard shortcuts for efficient notebook work.

Learn regression analysis as a prediction method for causal relationships, factor analysis, and data science. Begin with linear regression, then multiple regression, and learn to build, interpret, and compare models.

Explore the linear regression model as a simple, causal linear approximation of Y based on X, with beta zero, beta one, epsilon, and y hat predictions from sample data.

Differentiate correlation from regression: correlation does not imply causation and measures degree of interrelation. While regression reveals causality and effect, see how the income–education example illustrates prediction differences.

Explore the simple linear regression model by examining the regression line, B0 (intercept) and B1 (slope), and understand how residuals measure the distance between observed values and y hat.

Install and verify core Python data science packages—NumPy, Pandas, SciPy, Statsmodels, Matplotlib, Seaborn, and scikit-learn—using Conda or pip, and learn their roles in arrays, data frames, visualization, and regression.

Learn to perform simple linear regression in Python using pandas and Statsmodels, load data from a CSV, visualize with Matplotlib, and plot the regression line to predict GPA from SAT.

Learn to apply Seaborn styles atop Matplotlib to beautify graphs by importing Seaborn as sns and calling sns.set, while coding with Matplotlib.

Read statsmodels regression tables to derive the equation y_hat = 0.275 + 0.0017 x, and assess significance with standard errors, t statistics, and p-values.

Define SST, SSR, and SSE within the ANOVA framework to split total variability into explained and unexplained parts, and show SST equals SSR plus SSE.

Learn how ordinary least squares estimates the linear regression line by minimizing the sum of squared errors, identifying the intercept and slope (B0 and B1) for the GPA prediction.

Learn how R-squared measures the goodness of fit by comparing the variability explained by the regression to the total variability, with values from 0 to 1 and context across fields.

Apply the multiple regression model with several independent variables and their coefficients to compute y-hat, including the intercept, and minimize the sum of squared errors to improve explanatory power.

Explain how adjusted r-squared refines r-squared by penalizing excessive variables in regression analysis and compares models with the same dependent variable and data set, dropping useless predictors.

Discover the F statistic and F-test for linear regression. Learn how to test the overall significance of the model and interpret p-values to determine if all betas equal zero.

discover regression assumptions, including linearity, endogeneity of regressors, normality and homoskedasticity of the error, no autocorrelation, and no multi-collinearity, and learn why respecting them matters.

Assess linearity by plotting x against y on a scatter plot to check for a straight-line pattern; if non-linear, apply nonlinear regression or exponential and logarithmic transformations.

Explore no endogeneity and omitted variable bias, showing how missing relevant variables creates correlation between errors and regressors, with a London apartment example.

Explain normality, zero mean, and homoskedasticity of the error term, use the central limit theorem for large samples, and apply log transformations to fix heteroskedasticity and study elasticity.

Understand the no autocorrelation assumption, how serial correlation affects time series regressions, and why linear regression should be replaced by autoregressive, moving average, or ARIMA models.

Explore the no multicollinearity assumption in regression and how correlated predictors distort coefficient estimates. Learn fixes—drop a variable, or combine into a predictor, or include with caution—guided by correlation checks.

Encode categorical data with dummy variables in regression, mapping yes and no to 1 and 0. Predict GPA from SAT and attendance, illustrating two intercepts for attended vs not attended.

Learn to generate predictions with Statsmodels using a prepared data frame, including a constant, SAT, and attendance, and compare the effects of dummy variables on predicted GPA.

Explore Sklearn, a fast machine learning package for Python built on NumPy, SciPy, and Matplotlib, offering regression, classification, clustering, and dimensionality reduction with strong documentation and numerical stability.

Explore building a simple linear regression with sklearn to predict GPA from SAT scores, including data loading, feature setup, reshaping inputs, and fitting the model.

Learn how to interpret sklearn’s linear regression parameters, compare standardization and normalization concepts with default settings, and use fit, intercept, and predict to forecast GPA from SAT scores.

Load the dataset and apply multiple linear regression with sklearn, using SAT and rand one, two, three to predict GPA, then interpret the coefficients and intercept.

Explore F-regression in scikit-learn for feature selection by obtaining p-values for each feature, interpret univariate p-values, and note limitations for multivariate models.

Create a summary table with features, coefficients, and p-values to identify redundant variables, noting p-values reveal redundancy but don't measure overall usefulness.

Standardize numerical data by subtracting the mean and dividing by the standard deviation for feature scaling, yielding a mean of zero and a standard deviation of one.

Apply standardization to regression inputs to reveal true feature impact, using standardized coefficients (weights) and bias, enabling feature selection by identifying weights close to zero.

Predict values with a standardized model by scaling new data to the training mean and std, then use predict; compare GPA results from multiple and simple regression.

Explore underfitting and overfitting in regression, showing how models either miss data patterns or fit noise. Learn to use train-test splits to assess accuracy and prevent overfitting.

Learn to split data into training and testing sets using train_test_split, adjust test size and random state, and ensure consistent, shuffled or unshuffled data for aligned inputs and targets.

Explore a messy real-life car sales dataset, clean missing values, remove outliers using the 99th percentile, drop irrelevant features, and prepare data for regression analysis.

Apply the pandas get_dummies method to convert categorical features into dummy variables, dropping one category to create n minus one dummies and prevent multicollinearity.

Build a log-linear regression to predict log price from standardized inputs, with dummy features and an 80/20 train-test split, then evaluate via predictions, residuals, and R-squared around 75%.

We test our regression model by computing predictions with x_test, plot predicted used-car prices against a 45-degree line, and analyze residuals and percent differences to refine the model.

Explore logistic regression as a non-linear extension of linear regression for predicting binary outcomes. Learn how to model categorical results, use dummy variables, and forecast yes/no customer decisions.

Explore a simple logistic regression in Python using SAT and admitted data to predict admission probability. Compare linear regression and visualize the 0–1 bounded probability curve.

Explore how logistic regression uses the logistic function to model probabilities and odds for binary prediction, relate it to linear regression, and interpret the logit form and coefficients.

Build a logistic regression with Statsmodels, load data, map entries to zeros and ones, define dependent and independent variables, add constant to the predictor, fit with logit, and review summary.

An optional, very short coding tip explains debugging a regression summary when statsmodels and scipy break after an API change, with a GitHub discussion and a simple workaround.

Explore maximum likelihood estimation, compare models with log likelihood and the null model, apply the log likelihood ratio test, and interpret McFadden's pseudo R-squared in logistic regression.

Interpret the logit model by converting coefficients to odds with the exponential, revealing the odds ratio for unit SAT score changes.

Explore how binary predictors like gender are coded in logistic regression and interpret odds ratios, then improve models with sat scores to compare admission odds by gender.

Explore how to assess classification accuracy by comparing predicted probabilities to actual outcomes, using a confusion matrix and a simple accuracy calculation with statsmodels pred_table, including the 0.5 threshold.

Explore underfitting and overfitting in predictive models, learn how simple linear models underfit while overfitted models capture noise, and practice using training–test splits and confusion matrices to gauge accuracy.

Test the model in statsmodels on a 90/10 split test set, build a manual confusion matrix, and report 89.47% accuracy, noting training accuracy often exceeds test accuracy due to overfitting.

Explore cluster analysis as a multivariate method that groups observations by features, forming clusters by geography or language, and learn to select features and interpret results with scikit-learn and pandas.

Explore how clustering enables market segmentation and image segmentation, using scatter plots to identify customer segments and color-based image clusters, while highlighting clustering as an exploratory, business-focused technique.

Differentiate classification from clustering in supervised learning, where classification uses labeled data to train models for predicting output categories, and clustering groups data without labels.

Explore Euclidean distance and centroid concepts to perform clustering, measuring the distance between points in n-dimensional space using the Pythagorean theorem and computing the mean position.

Explore how K-means clustering identifies groups in data by selecting seeds, assigning points by proximity using Euclidean distance, and iteratively updating centroids to form two or three clusters.

Cluster countries by geographic location using k-means in Python with scikit-learn. Load data with pandas, plot clusters with matplotlib and seaborn, and explore different cluster counts.

Encode categorical language data as numbers, select features, and apply k-means clustering to reveal language-based and geography-based clusters, illustrating effects of mixed data on clustering.

Learn how the elbow method uses within cluster sum of squares (wcss) and inertia in k-means to select an optimal number of clusters, typically three in this example.

Explore the pros and cons of k-means clustering, including its simplicity and speed. Understand initialization sensitivity, outliers, elbow method for choosing k, and the role of Euclidean distance.

Explore how standardizing features affects clustering with k-means, illustrating scale's impact on groupings and when standardization should be used or avoided based on variable importance.

Use clustering to reveal omitted variable bias and improve regression by incorporating factors like location alongside apartment size to better predict price.

Explore market segmentation with k-means on a retail dataset, emphasizing standardizing satisfaction and loyalty, visualizing clusters, and using the elbow method to choose cluster count.

Explore market segmentation with k-means on standardized data, compare elbow method options, and interpret clusters like alienated, supporters, roamers, and fans across 2–5 cluster solutions.

Explore how clustering supports exploratory data analysis, market segmentation, and identifying groups like fans, supporters, roamers, and alienated customers to boost satisfaction and loyalty through targeted strategies.

Explore hierarchical clustering, including agglomerative and divisive methods, illustrated by the animal taxonomy example and a dendrogram; compare with flat clustering like k-means using Euclidean distance.

Explore how a dendrogram reveals hierarchical clustering on country data, showing how clusters form, merge, and how distance signals similarity to guide cluster counts.

Explore heatmaps and dendrograms with seaborn to perform hierarchical clustering on country data, comparing latitude and longitude patterns and identifying three clusters using Ward's method.