The Supervised Machine Learning Bootcamp

Install the Anaconda distribution to get Python, Jupyter Notebook, and data science packages, choosing your operating system and Python 3 64-bit version for a smooth setup.

Take a tour of the Jupyter dashboard, managing files and folders, uploading notebooks, and opening items. Use the interactive shell to write code and view output with IPython notebook formats.

Master the Jupyter dashboard by using code and markdown cells, executing with Ctrl+Enter or Shift+Enter, and leveraging keyboard shortcuts to edit, move, and navigate input and output fields.

Install and verify the key data science packages for the bootcamp, including NumPy, Pandas, Matplotlib, Seaborn, glob, and scikit-learn, using conda or pip for data manipulation and visualization.

Explore bayesian inference and naive bayes as fast, supervised classifiers for binary and multiclass tasks, with real-time spam filtering and sentiment analysis examples.

Explore how naive Bayes probabilities update beliefs with new knowledge through a thought experiment using red and blue balls, illustrating Bayes' theorem.

Explore Bayes' theorem and its product rule, linking joint and conditional probabilities of hypothesis and evidence. Identify priors, likelihoods, and posteriors, and see a deck-of-cards example illustrating a normalization constant.

Explore the naive Bayes approach to classifying messages as ham or spam by comparing word counts and conditional probabilities for real-time filtering.

Load five youtube dataset csv files and combine into one data frame. Drop nonessential columns, check nulls, balance ham and spam, and train a naive bayes classifier.

Learn how the count vector riser tokenizes strings, builds a vocabulary of tokens, and uses fit and transform to produce a sparse matrix of word frequencies.

Split the YouTube comments into train and test with 20% test size and stratified targets, then vectorize the training data with a count vectorizer for a naive bayes classifier.

Apply the naive bayes classifier to the YouTube dataset, exploring Gaussian, Bernoulli, categorical, multi nominal, and complement variants and comparing their suitability for text classification.

Learn to build and interpret a confusion matrix for a YouTube dataset, using confusion matrix display class to compare predictions with labels and visualize true positives and true negatives.

Explore how accuracy, precision, recall, and the F1 score quantify classifier performance using a confusion matrix and classification report for ham versus spam.

Adjust priors in the naive Bayes model to compare ham and spam predictions, observe the impact on the confusion matrix, recall, and overall accuracy for a YouTube dataset.

Explore the intuition behind k-nearest neighbors classification, using a three-neighborhood example to illustrate majority voting and distance-based assignment in supervised learning.

Explore distance metrics in two dimensions using the Minkowski framework, including Manhattan distance and Euclidean distance. Generalize to higher dimensions, connect to the Pythagorean theorem, and preview CNN applications.

Generate a random dataset using make_blobs or make_classification and create inputs with two features and a target. Assemble them into a Pandas data frame for analysis and future visualization.

Visualize a random dataset by scatterplots with hue by class, customize colors and markers, and ensure reproducibility to prepare for a subsequent classification algorithm.

Train a CNN classifier on a random 3-class dataset, perform a 20% stratified train-test split with a random state, and evaluate a k-nearest neighbors classifier with one and two neighbors.

explains how the knn classifier resolves ties with uniform weights using the mode, revealing index bias; then shows distance weighting with inverse distance to favor closer points.

Explore how different k values shape decision regions in a k-nearest neighbors classifier. A single neighbor yields volatile boundaries and overfitting; higher k smooths boundaries but increases bias.

Explore how choosing k from 1 to 50 affects misclassification rate, comparing uniform and distance-weighted weights to balance overfitting and underfitting.

Tune a k-nearest neighbors classifier with grid search and cross-validation, comparing uniform and distance weights to find the best n_neighbors for accuracy. Confirm the best model with a 0.95 accuracy.

Evaluate a knn model using a confusion matrix and classification report on test predictions, with 10 neighbors and uniform weights, for a three-class dataset; analyze precision, recall, and F1.

Apply k-nearest neighbors regression to a generated linear dataset, visualize data with matplotlib and seaborn, and predict y from x using 1–4 neighbors, illustrating the averaging mechanism.

Compare linear regression with k-nearest neighbors regression to contrast parametric and non-parametric approaches, showing how a linear dependency favors parametric models and affects overfitting with different k.

The lecture compares linear regression and KNN on a nonlinear data set (y = x^2 + sign(5x)), showing KNN with K≈7 outperforms linear models and highlights overfitting and underfitting trade-offs.

Explore the pros and cons of k-nearest neighbors for classification and regression, including distance-based predictions, non-parametric nature, and training, with notes on the curse of dimensionality, dataset size, and standardization.

Explore decision trees, from building blocks like root, nodes, and leaves to pruning techniques, with Python and Esk Learn, and see how they contrast with random forests.

Explore decision trees, a versatile flowchart-like data structure used across operations, research, and decision analysis to reach decisions through yes/no questions, with nodes, branches, and leaf outcomes.

See how well crafted decision trees become machine learning models through supervised training on data, using features, questions, and depth to enable classification and regression with ID3, C4.5, CART.

Explore the pros and cons of decision trees, a white-box, interpretable model with built-in feature selection, minimal preprocessing, and fast prediction, while addressing overfitting, pruning, and instabilities.

Implement decision trees in esc learn to solve a classification problem with the iris dataset. Explore 150 samples across three classes using sepal and petal measurements to classify iris species.

Import numpy and the iris dataset, prepare X and Y, train a decision tree classifier in a jupyter notebook, and use predict to classify iris flowers.

Plot a decision tree for iris classification using plot_tree with the math plot lib, and customize with feature names and class names, while interpreting root and leaves.

Discover how decision trees use splits evaluated by Gini impurity and information gain, based on class distributions and misclassification risk. Learn why minimizing impurity and balanced data matter for splits.

Explore decision tree metrics such as information gain and entropy, and compare them with Gini impurity, noting entropy's computational cost and the option to switch via the criterion parameter.

Explore how pruning reduces overfitting in decision trees by removing unnecessary subtrees and improving test accuracy, using minimal cost complexity pruning and the alpha parameter.

Discover how random forests use an ensemble of decision trees trained with bootstrapping to boost accuracy for regression and classification, with majority voting determining the final prediction.

Explore bootstrapping by uniformly sampling from the original training set with replacement to create diverse, same-size data sets that yield different decision trees in random forests.

Explore bootstrapping and bagging to build a random forest of decision trees, using majority voting and random feature subsets to reduce overfitting.

Learn to implement a random forest in Python using the glass dataset, including data prep, train-test split, model training, and evaluating with precision, recall, and accuracy.

Clean census income data by dropping entries with unknown values in work class, occupation, and native country, then one-hot encode features for decision tree and random forest modeling.

Train a decision tree classifier, visualize the tree, and diagnose overfitting; apply pruning using CCP alpha to improve accuracy from ~80% to ~84% and prepare for random forest.

Train a random forest on census-based income prediction, adjust estimators and prune with ccp alpha, and evaluate using the classification report for accuracy, precision, and F1.

Learn how support vector machines classify and regress data using hyperplanes and kernels for linear and nonlinear patterns. Understand strengths and tradeoffs, including interpretability, training time, and cross-validation.

Discover how support vector machines find the maximal margin hyperplane to separate fraudulent versus non-fraudulent transactions, using support vectors and hard or soft margin concepts.

Apply a soft margin in support vector machines to non-linearly separable data, allowing points inside the margin while maximizing separation; tune C via cross-validation to balance misclassifications.

Explore kernels for support vector machines to enable non-linear data separation using Gaussian, polynomial, linear, and sigmoid kernels via the kernel trick.

Code a Python-based support vector classifier on the mushroom dataset to predict poisonous or edible mushrooms, loading with pandas and encoding categorical features, with grid-search cross-validation and a confusion matrix.

Standardize the data to ensure equal feature weight and prevent bias, then split inputs and targets into train and test sets with 80/20 ratio and stratify by target for balance.

Encode categorical features with label and ordinal encoders for x and y, then scale inputs to -1 to 1 with a minimax scaler before training a support vector classifier.

Implement a linear support vector machine to classify data by fitting on training data, with C=1. Apply it to test data scaled to -1 to 1 and preview metrics.

Analyze a linear support vector classifier using a confusion matrix with edible and poisonous labels, and evaluate precision, recall, and F1 to improve poisonous mushroom recall through cross-validation.

Learn how cross-validation uses training and validation splits to evaluate models, tune hyperparameters, and avoid bias, including five- and ten-fold, and leave-one-out approaches.

Optimize a support vector classifier by tuning kernels, C values, and gamma through cross-validation and grid search to reduce misclassifications and improve model performance.

Apply grid search cross-validation to tune a support vector classifier, testing polynomial kernel with C=1 or 10, and evaluate with precision, recall, and F1 on development and test sets.

Explore regression analysis as a supervised learning approach for predicting numeric outcomes with a dependent variable and predictors, using linear, polynomial, ridge, and lasso methods to prevent overfitting.

Explore how overfitting arises when models capture noise in training data and how multilinearity among predictors undermines reliability on unseen test data, then learn how regularization helps prevent these issues.

Explore how regularization prevents overfitting in regression by shrinking coefficients through L1 (lasso) and L2 (ridge) penalties, balancing bias and variance for simpler, more generalizable models.

Explore ridge regression as a regularization technique that reduces overfitting by penalizing the magnitude of coefficients with a tuning parameter lambda. Use cross-validation to choose the bias-variance trade-off.

Explore ridge regression as a regularization method that adds a penalty term to the least squares objective, tuning lambda via cross-validation to balance bias and variance and improve generalization.

Analyze a two-group house price model using ridge regression, adding a penalty to shrink size differences and stabilize predictions, then preview the lasso approach.

Learn how lasso regression uses L1 regularization to shrink coefficients, possibly to zero, enabling feature selection and simpler models. Compare it to ridge, tune lambda to balance bias and variance.

Compare ridge and lasso regression, showing ridge uses L two norm regularization to shrink coefficients and prevent overfitting, while lasso uses L one norm regularization enabling feature selection via cross-validation.

Explore ridge and lasso regression with the Hitters dataset, preprocessing categorical features via get dummies, handling nulls, and preparing data for regression models to predict salary.

Explore exploratory data analysis, assess salary distribution and correlations with player statistics, visualize feature relationships with a heat map, and learn how ridge and lasso regularization handles correlated features.

Learn to build a linear regression model by selecting predictors, standardizing features, splitting data into training and testing sets, and evaluating with mean squared error and R squared.

Use cross-validation to select the lambda tuning parameter for ridge regression by splitting training data into five folds and choosing the lambda with the lowest validation error.

Explore ridge regression with repeated k-fold cross-validation to tune the alpha parameter and predict salaries from the hitter dataset using Python and cross-validated models.

Apply lasso regression with cross-validation in Python, observe coefficient shrinkage to zero, and evaluate performance with RMSE and r-squared, comparing to ridge.

Compare three regression models—linear, ridge, and lasso—using training and testing scores, r squared, and RMSE, highlighting ridge as the best performer on test data and the value of regularization.

Replace missing salary values in the hitter dataframe by using ridge regression to predict and fill the gaps. Scale predictors, generate predictions, and update the dataframe with the estimated salaries.