Data Science: Supervised Machine Learning in Python

Explore the intuition of k-nearest neighbors for classification, using distance to identify the closest examples and vote to predict the outcome.

Explore the k-nearest neighbor algorithm for supervised learning in Python, using training data to predict labels by majority vote among the closest points, with k as a tunable hyperparameter.

implement k-nearest neighbors on the mnist dataset, using a 1000 training and 1000 test split, and compare test accuracy for k values 1 through 5.

Explore how k-nearest neighbors can fail on a grid of alternating dots, where a three-nearest-neighbor vote consistently predicts the wrong class, exposing the geometry and weaknesses of the model.

Explore how k-nearest neighbors solves the XOR problem in Python by modeling a nonlinear boundary, generating a 0 to 1 grid, and evaluating accuracy with k=3.

Apply knn to the donut problem in supervised machine learning with Python. Setup uses inner and outer classes, gaussian noise, and polar coordinates, shown via a scatterplot with perfect accuracy.

Learn how different k values shape k-nearest neighbors classifiers, using cross-validation to balance bias and variance and avoid overfitting for unseen data.

Improve the efficiency of the k nearest neighbors algorithm by replacing loops with optimized vector operations like numpy dot and consider Katy trees and dimensionality reduction for faster distance calculations.

This lecture introduces a suggestion box for student feedback on the course, inviting detailed input on background, course difficulty, missing topics, and future course ideas to improve learning.

Explore how a Bayes classifier uses Gaussian distributions, probability, and Bayes' rule to classify continuous features like height and weight, handling the prior and log probabilities.

Explore the Bayes classifier for discrete distributions in text classification, using spam detection to compare binomial and Bernoulli models with priors, likelihoods, and the posterior.

Learn how Naive Bayes uses independence to model p(words|class) and p(class), predicting spam vs not spam, and apply gaussian approaches with axis-aligned covariance and log probabilities.

Explore a manual naive bayes example using a spam classifier with words money, free, and pills; compute independent word probabilities and posteriors, and apply add-one smoothing to avoid zero probabilities.

Code and evaluate a gaussian naive bayes model on MNIST in Python, implementing fit with smoothing, priors, log-likelihood, and a score method while comparing to k-nearest neighbors.

Explore non-naive bayes by modeling feature interactions in p(x|c) with full covariance and multivariate Gaussian, contrasting with naive bayes independence and exploring complex alternatives.

Upgrade the bayes classifier for MNIST by using covariance instead of variance, transposing input, applying an unbiased n-1 normalization, and implementing two code changes for better results.

Learn how to use Bayes rule with Gaussian class distributions to derive quadratic discriminant analysis and, when covariances match, linear discriminant analysis for binary classification.

Compare generative and discriminative classifiers by modeling data differently: generative models estimate P(x|C) for each class and apply Bayes rule, while discriminative models estimate P(C|x) directly.

Explore the intuition of decision trees as a sequence of if statements, illustrated by a disease classifier using BMI and blood pressure, and discuss threshold optimization and arbitrary-depth nesting.

Explore how decision trees function as classifiers built from nested if statements with axis-aligned splits. Learn how leaves and left or right predictions drive binary classification.

Apply information entropy to measure uncertainty and optimize decision tree splits, noting that maximum entropy occurs at p = 0.5 for binary variables.

Explore how information entropy and information gain guide attribute selection and data splits in decision trees, using thresholds for continuous data and depth-based leaf nodes to control overfitting.

Learn how to choose the best split for continuous features by evaluating midpoints between ordered points, identifying boundaries where labels differ, and maximizing information gain based on entropy.

Demonstrate a decision tree in code by implementing entropy, information gain, and recursive fitting to split data into left and right and predict labels.

Explore the perceptron, a historical binary classifier, and its training rule. See how misclassified samples update weights with a learning rate and how bias is absorbed into weights.

Write a Python perceptron to learn a linearly separable dataset, updating weights with the learning rate on misclassified samples, and plot cost while aiming for 100% train and test accuracy.

Learn how a perceptron learns binary classification with MNIST and XOR data in Python, including target encoding to -1 and +1, accuracy results, and the model's limitations.

Explore the perceptron loss function for supervised learning in python, showing how misclassified samples increase loss and how stochastic gradient descent minimizes it by adjusting the weight vector.

Explore hyperparameters in supervised machine learning and learn how k-fold cross-validation, with typical values like 5, 8, or 10, uses training and validation splits to optimize and improve generalization.

Explore feature extraction and feature selection in supervised machine learning, using the radius feature to separate data, discuss domain knowledge, and compare greedy selection with pca.

Compare supervised machine learning models with deep learning, weighing training time, prediction speed, and memory. Explain how linear methods and decision trees remain interpretable, while neural networks offer nonlinear boundaries.

Discuss multiclass classification in supervised machine learning with python; compare strategies like one-vs-rest and one-vs-one, highlight training inefficiency and class imbalance, and use scikit-learn for built-in multiclass models.

Explore scikit-learn, a Python machine learning library offering classifiers and regressors with a consistent API. Learn about probabilistic outputs, ROC AUC, thresholds, and saving models with pickle.

Learn regression with scikit-learn using k-nearest neighbors and decision tree regressors on a sine wave. See how kNN yields blocky predictions and how distance weighting can smooth results.

Build a machine learning web service that trains a model on handwritten digits and serves predictions through a post endpoint using a tornado-based API.

Train a classifier on 25 percent of the data and save the model with pickle. Run a tornado-based web service on port 8888 to handle predictions via a get/post API.

Explore next directions in supervised machine learning with support vector machines and ensemble methods like random forests and Ex-GI boost, highlighting how ensembles improve accuracy over single models.

Explain the appendix, renamed the faq, as optional supplementary videos that answer common questions; use the q&a to get help and align with course concepts.

Installations are presented as guidelines that emphasize understanding principles over syntax and Python prerequisites, with guidance on using pip to install libraries like OpenAI Gym and avoiding deprecated tools.

Install data science libraries on Windows with Anaconda, isolate environments, and install NumPy, pandas, scikit-learn, TensorFlow, Keras, PyTorch, and OpenAI Gym for reinforcement learning.

Learn how to set up a data science development environment across Windows, Linux, and Mac, installing numpy, scipy, matplotlib, scikit-learn, and tensor flow, with virtual machines or native installers.