Mastering Machine Learning from Scratch

Explore supervised learning by modeling regression with continuous targets, training regression models on data, and predicting unseen outcomes using simple, multiple, polynomial, and ridge, lasso, elastic net.

Learn how linear regression fits the best fit line to data, minimizes the sum of squared residuals, and predicts umbrella sales from rainfall with y_hat = beta0 + beta1 x.

Learn to derive the best fit line in linear regression by minimizing the sum of a square error using derivatives and gradient descent intuition for beta naught and beta 1.

Use gradient descent to minimize mean squared error in linear regression by updating m and b. Alpha controls step size; ensure linearity, residuals near zero, homoscedasticity, and no multicollinearity.

Master practical multiple linear regression in Colab, transforming states into dummy variables, performing train-test split, and applying min-max scaling for robust predictions.

Train a linear regression model in two lines of code, fit it on scaled data, and interpret intercept and coefficients to predict and evaluate with MSE, RMSE, and R2.

Explore how the 80/20 train and test data split and model complexity affect bias, variance, and overfitting, and balance underfitting and overfitting through the bias-variance tradeoff.

Explore how regularization tackles overfitting by constraining coefficient sizes, balancing bias and variance to an optimal model complexity, and compare L1 (lasso), L2 (ridge), and elastic net.

Explore L1 regularization, or lasso regression, which penalizes the absolute value of coefficients to shrink features to zero, effectively performing feature selection via the regularization parameter lambda.

This lecture explains L2 regularization, or ridge regression, which adds a squared-coefficient penalty to shrink coefficients without forcing them to zero, useful for correlated features and preventing overfitting.

Explore cross-validation, including fourfold splits, to train on 75% and test on 25% of data, and address overfitting and underfitting with L1/L2 and elastic net regularization and hyperparameter tuning.

Apply regularization in linear regression by using lasso, ridge, and elastic net, evaluate with mean square error, and tune alpha via cross-validation and grid search.

Master polynomial regression, extending linear regression with polynomial features to capture non-linear relationships. Use scikit-learn to compare degree two performance against linear regression and avoid overfitting.

Explore evaluating regression models with gradient descent insights and metrics like mean squared error, RMSE, MAE, and R-squared; learn how data context and adjusted R-squared guide hyperparameter-aware comparisons.

Explore the shift from regression to classification, focusing on binary classification with logistic regression and the sigmoid output. Apply thresholding to convert probabilities into labels and learn with binary cross-entropy.

Master logistic regression from theory to practice by learning the sigmoid s(theta), cost minimization via gradient descent, and evaluating with confusion matrix, accuracy, precision, recall, f1, and ROC AUC.

Implement logistic regression to classify churn on a real-world dataset, transforming categorical features with label and one-hot encoding, scaling with min-max, and evaluating with accuracy, confusion matrix, and ROC-AUC.

Explore how the decision tree, a supervised learning algorithm for classification and regression, splits data from the root node to leaf nodes to make predictions.

Explore how a decision tree splits data for classification, using information gain and entropy to create homogeneous subnodes, and compare with Gini and regression criteria.

Apply a practical decision tree to the loan df dataset using label-encoded features. Explore data cleaning, correlation heat map, and hyperparameter tuning, then evaluate with ROC AUC and confusion matrix.

Explore ensemble learning, a technique that combines multiple models to improve accuracy and robustness, reduce overfitting, and diversify predictions through bagging, boosting, and stacking.

Mastering machine learning from scratch introduces boosting as a sequential ensemble method that weights misclassified examples to improve subsequent models, contrasting it with bagging’s parallel approach.

Understand stacking, where a meta model uses predictions from multiple base models as features to produce a final prediction, illustrating ensemble learning across bagging, boosting, and stacking.

Explore how random forest, a bagging ensemble of multiple decision trees, achieves high accuracy in classification and regression via random feature selection, bootstrap sampling, and majority voting or averaging.

implement and evaluate a random forest classifier on a credit default dataset, using train-test split, grid search cv, and roc auc to tune hyperparameters and assess performance.

XGBoost extends extreme gradient boosting with regularization, parallel processing, and early stopping, building sequential trees that prune splits, handle missing values, and optimize via gradient descent with a learning rate.

Apply an XGBoost classifier to a credit dataset, performing train and test split and grid search CV to tune hyperparameters, then evaluate with ROC AUC, precision, recall, and F1.

Explore non-linear learning with the k-nearest neighbor algorithm, a supervised instance-based method for classification and regression using distance-based voting and practical considerations like scaling and k selection.

Master the support vector machine for classification by defining the hyperplane that maximizes margin using support vectors. Explore kernel tricks for non-linear separation with linear and rbf kernels, in Colab.

Explore how unsupervised learning trains on unlabeled data with no target variable to uncover patterns, clusters, dimensionality reduction (PCA), anomaly detection, and applications like customer segmentation and topic modeling.

Explore how the elbow method uses within-cluster sum of squares to pick the optimal k in k-means, highlighting centroids, cluster variation, and the silhouette score.

Mastering machine learning from scratch introduces the silhouette score to determine the optimal number of clusters in k-means by balancing cohesion and separation, complementing the elbow method.

Explore hierarchical clustering, an unsupervised technique that builds a dendrogram to reveal nested data groups and determine optimal clusters using agglomerative and divisive approaches.

Apply k-means and agglomerative clustering to unsupervised learning, using scaling, elbow and silhouette analysis to determine optimal clusters, and compare cluster quality with dendrograms.