Python Machine Learning Bootcamp

Explore supervised learning, including classification with labeled data and regression with numerical targets, using true values to train models and tune algorithms.

Explore classification with logistic regression as the base classifier, and learn how to assign data to categories and groups in binary, multilabel, and multiclass tasks.

Logistic regression uses weights and a bias applied to features to produce y hat via a sigmoidal function; minimize log loss by adjusting weights through the cost function and derivatives.

Explore how gradient descent optimizes logistic regression by minimizing log loss, updating weights with learning rate, and choosing between batch, stochastic, and mini-batch approaches.

Explore binary, multiclass, and multilabel classification in machine learning, and compare one-versus-rest and one-versus-one strategies. Learn how multinomial loss, cross entropy, and dataset balance affect model performance.

Learn how to build a binary classifier with logistic regression using the iris data, create train-test splits, train, predict, score accuracy, and visualize decision boundaries.

Learn to build and train a multiclass classifier with logistic regression, using stratified shuffle split and solver choices to switch between one-versus-rest and multinomial approaches.

Explore how classifiers make predictions and evaluate them with accuracy, precision, recall, F1 score, and precision-recall curves; compare hard and soft voting and study confusion matrices and support.

Learn precision and recall, including true positives and false positives, and how thresholds shape them. Use F1 score and precision-recall curves to balance accuracy in disease detection and fraud detection.

Use the ROC curve to compare classifiers via true positive rate vs false positive rate and the AUC, and interpret the confusion matrix to reveal errors and understand support counts.

Explore the MNIST handwritten digits dataset by loading it into a data frame, inspecting features and targets, and visualizing sample grayscale images to understand pixel-level classification.

practice evaluating classifiers in Python by transforming targets for binary classification, using stratified train-test splits, and assessing accuracy, precision, recall, F1, and confusion matrices.

Master using a validation set to evaluate a model during training and understand generalization error, with stratified shuffle splits and a held-out test set, measuring accuracy, precision, recall, and f-score.

Explore cross-validation with stratified splits to train and evaluate a model multiple times, using logistic regression to measure accuracy, roc auc, and f1, and review train vs validation variability.

Explore hyperparameters and how to tune them to improve model performance within cross-validation, including tolerance, fit intercept, class weight, max iterations, solver, and parallel training.

Explore regularization theory by adding a loss term to control overfitting and improve generalization, including L1, L2, elastic net, and early stopping.

Explore the sources of generalization error, including overfitting, underfitting, irreducible noise, and the bias-variance trade-off, and see how training versus validation curves reveal these effects.

Explore practical regularization in logistic regression by selecting penalties (L2, L1, elastic net or none), tuning C, and understanding solver compatibility.

Learn to tune hyperparameters with grid search cross-validation and randomized search, using logistic regression options like penalty and C, while managing results, scoring, and refit for best models.

Clean and prepare input data by handling missing values with imputation and threshold-based row/column dropping; compare simple mean/zero imputation with nearest-neighbor imputation and custom methods for better model performance.

Learn how feature scaling standardizes data for machine learning, using min-max, standardization, log transforms, percentiles, and thresholds—handling outliers and applying to tabular and image data to improve model performance.

Apply feature scaling in practice using min-max scaling, standardization, log transformation, and percentile-based thresholds with logistic regression. Discuss evaluation on training and validation sets with cross-validation to assess performance changes.

Learn to handle text and categorical data with Titanic data to predict survival. Apply ordinal encoding with defined categories and use one-hot encoding for nonordinal features, while handling missing values.

Master transformation pipelines to streamline numeric data preprocessing, including imputation and scaling, fit a logistic regression, and ensure consistent training and testing transformations.

Design a custom transformer class with fit, transform, and fit_transform, leveraging the sklearn transformer mixin; vectorize a function for efficient transformation and reshape data for pipeline compatibility.

Build and apply column-specific pipelines with a column transformer to transform numerical, cabin, and text features using custom transformers, ordinal encoding, and one-hot encoding.

Learn to handle imbalanced data using under sampling and oversampling with imbalanced-learn, compare metrics like recall, precision, F1, and AUC, and interpret confusion matrices for fraud detection.

Investigate feature importance by inspecting logistic regression coefficients, visualize weight magnitudes, and reduce features to improve training time while preserving accuracy.

Save and load trained models and pipelines with joblib or pickle, ensuring data passes through the same training pipeline; evaluate on the test set and consider retraining on full data.

Prepare and deploy machine learning models beyond prototyping by coordinating with engineers, saving models to scripts, and ensuring production readiness through monitoring, retraining, and human feedback.

Explore multilabel classification by predicting three targets (the main label, number of loops, and has loops) using a multi output classifier with logistic regression, and a custom multilabel class.

extend linear models with polynomial features to capture nonlinearity in the Make Moons dataset, using logistic regression and degree-three terms, while cautioning about feature explosion and overfitting.

Explains how support vector machines find the maximum margin hyperplane using support vectors and hinge loss, including hard and soft margins, kernel options, and feature scaling.

Learn practical SVM implementations on the iris dataset, comparing linear SVM and linear SVC with kernels, and visualize decision boundaries and support vectors.

Explore the K nearest neighbors algorithm, a non parametric model that classifies by voting among the k closest instances using Euclidean or Manhattan distances, with option for weighted votes.

Apply k-nearest neighbors classification to the iris dataset, using two features for visualization and plotting the decision boundary while evaluating training and testing scores.

Learn how decision trees classify by traversing nodes and leaves, using thresholds like petal length and width on the iris dataset, and how cart algorithm and impurity measures guide splits.

Explore decision tree pruning using minimal cost complexity pruning to balance misclassification and leaf count, tune alpha after training to reduce overfitting and improve generalization.

Practice practical decision tree implementation on IRS data with training/testing splits, entropy impurity, maximum depth and minimum samples, and observe random_state effects on feature importances and boundaries.

Explore random forest theory by training a diverse ensemble of decision trees, using bagging and voting to improve accuracy while leveraging out-of-bag scoring for generalization.

Explore practical random forest implementation using sklearn, building an ensemble of decision trees, tuning n_estimators, bootstrap, max_samples, and pruning, and evaluating with out-of-bag scoring and visualizations.

Master the Naïve Bayes theory, a simple Bayes model using independence and Bayes theorem, with additive smoothing for unseen values and training data classification in spam and text tasks.

Explore how to implement gaussian naive bayes and multinomial naive bayes, set priors, apply variance smoothing, fit on training data, and visualize decision boundaries.

Learn to choose a machine learning model by weighing white box versus black box complexity, test contenders with cross-validation and grid search, and leverage maps and peer research.

Learn how regression predicts numerical values with linear regression and convex loss. Explore mean squared error, mean absolute error, and regularization such as ridge, lasso, and elastic net.

Demonstrate practical linear regression by predicting petal width from length and class in a two-feature iris dataset, and explain intercept, normalization, and feature scaling in a pipeline.

Explore regularized linear regression in python with lasso, ridge, and elastic net penalties using coordinate descent. Build features with one-hot encoding, pipelines, and column transformers, and evaluate with r-squared score.

Load and inspect the Boston housing data for regression, examining features and the median value target. Perform exploratory data analysis with histograms, scatter plots, and correlation visuals.

Explore polynomial regression with the Boston housing data, comparing linear regression, lasso, ridge, and elastic net using degree-2 features, 5-fold cross-validation, and feature selection via non-zero coefficients.

Explore regression losses such as huber loss and epsilon-insensitive variants, and learning rate strategies from constant to adaptive, inverse, and power schedules to optimize models.

Explore stochastic gradient descent regression on the Boston housing data, examining various losses and learning-rate schedules, cross-validation, and regularization to understand how SGD regressor performance compares to linear models.

Explore KNN regression, which uses surrounding neighbors to estimate a numeric target by averaging their values, with options for uniform or distance-weighted weighting, as shown on the Boston housing data.

Explore knn regression on the Boston housing dataset using a pipeline with standard scaler, train/test split, and cross-validation, and assess how neighbor count and scaling affect performance and leakage.

Explore SVM regression theory, where the objective keeps the margin narrow while containing as many instances within the epsilon width around the margin, with kernels and shrinking to speed training.

Explore practical svm regression on Boston housing data using a scaled pipeline, cross-validation, and kernel comparison (linear, polynomial, rbf, sigmoid) with regularization and epsilon tuning.

Explore decision tree regression theory by adapting split criteria to mean squared error and mean absolute error, addressing overfitting with regularization and leveraging random forests.

Explore regression with decision trees and random forests using the Boston housing data, tuning max depth and mean squared error to balance underfitting and overfitting.

Explore regression metrics including mean squared error, mean absolute error, R-squared (coefficient of determination), and explained variance, and learn when to use each for model evaluation and scoring.

Explore ensembles by combining predictions from several models to improve accuracy when their errors are uncorrelated, and use weak learners to form strong ensembles like random forest.

Learn how voting ensembles combine multiple models for classification and regression, using hard and soft voting to aggregate predictions, improve accuracy, and apply mean or weighted mean across models.

Implement a voting classifier with logistic regression, decision trees, and naive Bayes, using hard or soft voting, weights, cross-validation, and stratified split for ensemble performance.

Demonstrates applying a voting regressor for a Boston housing regression task, combining k-nearest neighbors, ridge, and decision tree estimators to boost the R-squared score by averaging predictions.

Explore bagging and pasting as ensemble methods that use bootstrap aggregation and random sampling with or without replacement to create diverse models.

Demonstrate bagging and pasting with thirty decision trees in the ensemble, using eighty percent samples and ninety percent features, a stratified shuffle split, and out-of-bag scoring.

Apply bagging regression with decision tree estimators on the Boston housing data, including feature scaling and train-test splits. Improve performance through ensemble averaging and hyperparameter tuning.

AdaBoost theory and adaptive boosting fuse weak learners into a strong ensemble by exponential weight updates, focusing on mistakes to improve predictions with learning rate and regularization.

Explore practical AdaBoost classification by building a boosted ensemble with decision trees, tuning learning rate and base estimators, and evaluating stage-wise predictions and scoring through weighted aggregation.

Train an AdaBoost regression model on the Boston housing data using a pipeline with standard scaler and a 30-estimator decision tree base estimator, tuning learning rate and loss.

Learn gradient boosting, an ensemble of sequential weak learners that model residual errors, with techniques like early stopping, shrinking learning rate, and extras like bagging or extra boost.

Learn to implement gradient boosting for classification using a gradient boosting classifier on MLS data, tune estimators, loss, learning rate, and tree criteria, and assess stage-wise accuracy.

Apply gradient boosting regression with regression trees to form an ensemble. Tune loss-driven residuals, learning rate, and subsample to balance bias and overfitting, evaluating with r-squared.

Learn stacking and blending theory by training multiple models in a base layer, using a holdout set to train the second layer, and optionally adding more layers to improve generalization.

Demonstrates stacking and blending of classifiers with stratified train-test splits, using a base layer of decision tree, SVM, and logistic regression.

Learn stacking regression with a stacked ensemble, combining base estimators like decision tree and SVM using a lasso final estimator, with hyperparameter tuning and careful data splitting.

Explore how dimensionality reduction reduces high-dimensional data by projecting onto lower dimensions and reveals underlying patterns, guiding when to use projections or manifold learning.

Master principal component analysis, reducing dimensions by projecting data onto axes that maximize variance using eigenvectors and singular value decomposition. Explore kernel PCA, explained variance, and efficient hyperparameter tuning.

Explore practical PCA with scikit-learn, including fit_transform, transform, explained variance ratio, and inverse_transform for data reconstruction. Compare linear PCA and kernel PCA on real datasets using a random forest classifier.

Explore non-negative matrix factorization for dimensionality reduction by decomposing a data matrix X into W and H to reconstruct X with Frobenius norm and L1, L2, or elastic net regularization.

Learn practical non-negative matrix factorization with AMREF: set components, initialization, and solver, transform data to the W matrix, and achieve faster training with minimal accuracy loss.

Isomap theory preserves geodesic distances while reducing dimensions by building a nearest-neighbor graph, approximating the surface, computing a geodesic distance matrix, and applying SVD to project onto lower dimensions.

Explore practical Isomap on an s-curve dataset, project from 3d to 2d using nearest-neighbors and geodesic distances, and evaluate with a random forest classifier.

Learn how locally linear embedding uses local neighborhoods to approximate linear distances, reconstruct points from neighbors with a weight matrix, and project data into lower dimensions.

Explore practical implementation of locally linear embedding for manifold unfolding, comparing standard, modified, Hessian, and LDA methods, with 2d visualizations and a random forest classifier evaluating accuracy and training time.

Explain t-SNE theory, using joint probability and perplexity to preserve local structure when projecting high-dimensional data to two or three dimensions, noting non-convex optimization effects.

Master the practical t-SNE workflow by tuning perplexity, early exaggeration, learning rate, and initialization, then visualize two-dimensional embeddings and compare Barnes-Hut and exact modes.

Explore unsupervised learning for unlabeled data, using clustering to discover structure and anomalies to identify outliers, and combine with semi supervised learning to propagate labels.

Uncover the k means clustering algorithm and how it minimizes inertia by assigning each data point to the nearest centroid. Learn centroid updates, initialization strategies, and mini batch canings.

Explore practical k-means clustering with full and mini-batch implementations, data loading and stratified splits, initialization options, inertia, convergence, and visualizing cluster centers on toy datasets.

Learn how to choose the number of clusters using elbow and inertia trends, silhouette analysis, and metrics like adjusted rand index and vix measure, with iris and make blobs examples.

Assess cluster count using silhouette metrics (silhouette samples and silhouette score) on toy and amnesty data with k-means, and compare adjusted rand score, homogeneity, completeness, and V-measure.

Explore how dbscan finds clusters as connected dense regions using core samples, epsilon, and minimum samples. Identify core, non-core samples, and outliers, and see how these parameters control noise.

Explore practical dbscan clustering by tuning epsilon, min samples, and metrics, review core samples, clusters, labels, and understand why no predict method exists—then label new data with a supervised classifier.

Explore gaussian mixtures as unsupervised learning that models data with multiple gaussian distributions and unknown parameters. Apply expectation maximization, convergence, and AIC/BIC to select clusters and detect anomalies.

Explore practical implementation of gaussian mixtures using the moons dataset, including initializing parameters, visualizing component shapes, evaluating with AIC/BIC, and applying outlier detection via score_samples and sampling.

Explore semi supervised learning to leverage partially labeled data sets with label propagation and clustering, labeling key centers and then training a classifier with supervised learning and active learning.

Implement semi-supervised learning by using k-means to create cluster centers and label closest data points, then compare label propagation and label spreading with SVM and kNN on a reduced dataset.