Machine Learning Essentials - Master core ML concepts

Explore the basics of supervised learning, where labeled data reveals relationships between features and targets, covering regression for continuous outputs like house prices and classification for discrete labels.

Explore supervised learning by predicting student marks from study hours, train a model on training data with X and Y, and evaluate generalization on unseen test data.

Explore unsupervised learning, differentiate it from supervised learning, and master clustering with k-means and dimensionality reduction techniques like BCA and SVD on unlabelled data.

Master linear regression from a single feature to multivariate models, explore the error function and gradient descent, evaluate predictions using evaluation metrics, and apply to a house price challenge.

Explore linear regression with a single input feature to predict a target. See how training data X and y enable a linear model with theta to minimize error.

Explore how linear regression uses a hypothesis function h(x)=theta1 x+theta0 to predict marks from hours of study in a single-feature model, with theta1 and theta0 as learnable parameters.

Initialize theta randomly and iteratively reduce the mean squared error loss to train a linear regression model. Use gradient descent to converge toward the minimum error and better predictions.

Explore gradient descent as an optimization method to find function minima, using iterative updates with slope and learning rate, fundamental to linear and logistic regression and neural networks.

Demonstrate gradient descent on a quadratic function by updating x via x = x - learning rate times gradient, illustrating convergence to minimum at x = 5 and learning-rate effects.

Explore how gradient descent optimizes the linear regression loss J theta, with theta0 and theta1, by minimizing mean squared error to reach the global minimum.

Compute the gradient of the loss with respect to theta naught and theta one, and update each by subtracting the learning rate times its partial derivative.

Implement a linear regression workflow from data generation to building the hypothesis and training loop. Use numpy and matplotlib to generate X and Y, visualize data, and evaluate progress.

Master a train-test split, typically 80/20, training on 80% and testing on 20%, while shuffling data and evaluating predictions against true y values on unseen data.

Build a regression model by designing the hypothesis function and the mean squared error, then train with a gradient descent loop updating theta naught and theta one.

Interpret theta as the predictive line by fitting to training data and predicting test data via the hypothesis; plot train and test points, and visualize the line.

The R2 score, or coefficient of determination, measures how closely regression predictions match actual data, with one indicating a perfect fit and zero or negative values indicating poorer models.

Define r2 score using y and y_pred; compute numerator as sum of squared errors and denominator as sum of squared deviations from the mean; apply to test data yielding 0.95.

Visualize the training of linear regression with multiple features by plotting the J theta loss surface in three dimensions and its contours using mean squared loss.

Trace the trajectory of gradient descent on a linear regression model by recording theta values during training. Visualize the loss surface with 3D and contour plots to show convergence.

Explore linear regression with multiple features to predict housing prices using area, rooms, locality, and other factors, and learn the role of X, Y, theta, and the hypothesis y hat.

Formulate the hypothesis as a vectorized dot product of theta and x, including a bias term, enabling fast predictions via matrix-vector multiplication and avoiding loops.

Understand how the loss function measures prediction errors across examples using predicted minus actual values, while the hypothesis evolves with feature transforms, and use vectorized code with np.dot and np.mean.

Train the model with gradient descent to reduce loss. Update thetas via learning rate times the average gradient across examples, where the gradient equals (prediction minus actual) times the feature.

Learn to build a linear regression model from scratch using a synthetic dataset generated with make_regression, normalize features, visualize their impact, and perform a train test split.

After preprocessing the data for linear regression, this lecture builds the hypothesis using vectorized x theta multiplication with a bias term to produce predictions for all examples.

Compute the loss efficiently by predicting with theta and x, measuring mean squared error between predictions and actual values, squaring errors, and averaging them without loops.

Compute the gradient for training a linear model with vector code: gradient equals X^T (y_pred - y) divided by m, and update theta using the learning rate.

Implement a training loop by preprocessing X to add a column of ones, initialize theta, and perform gradient descent with a learning rate and max iterations, tracking the loss.

Explore why theta is an 11-element vector, compare matrix and vector shapes in numpy, and ensure compatible shapes through learning rate and gradient updates using np.dot.

evaluate a regression model with a fixed r-squared style score, using y mean, y test, and y predicted; see how reducing noise improves the linear model fit.

Learn to perform linear regression with a prebuilt library, train the model on x and y data, generate predictions, evaluate with R2 score, and inspect coefficients and intercept.

Explore logistic regression for binary classification, predicting probabilities between 0 and 1 and using thresholds to decide yes or no, with real-world examples like spam detection and loan default.

Explore logistic regression on a two-feature fruit dataset, learning a linear decision boundary to distinguish apples from oranges using a probability between 0 and 1, with thresholding at 0.5.

Explore how logistic regression uses a sigmoid of theta transpose x to produce a probability between 0 and 1, learning a decision boundary with theta weights over features.

Explore why mean squared error is unsuitable for logistic regression and how binary cross entropy, or log loss, offers a convex loss that links predictions to true labels.

Apply gradient descent to minimize the binary cross-entropy loss J(theta) by updating theta via the gradient update rule, using the sigmoid hypothesis on x until reaching a local minimum.

Prepare data for logistic regression by generating a toy dataset with make blobs and visualizing two clusters. Standardize to zero mean and unit variance and perform a train-test split.

Construct a logistic regression hypothesis using the sigmoid of theta transposed x, add a bias column, and generate predictions for all examples.

Learn to implement binary cross entropy loss with vectorized, elementwise operations on the actual labels y and the predicted probabilities, using log terms and mean reduction for efficient training.

Compute the gradient for the entire dataset to train the model, using the difference y − ŷ, X^T, and 1/m to form the gradient vector.

Write the training loop to train a model with x and y, using 100 iterations and a learning rate of 0.1, initializing theta randomly and applying gradient descent.

Visualize data in two dimensions by plotting the logistic regression decision boundary from theta0 + theta1 x1 + theta2 x2 = 0, sample training points, and assess accuracy with predictions.

Explore predicting with a hypothesis function by turning probability scores into final labels via thresholding, and measure training and test accuracy in logistic regression.

Master logistic regression with a sklearn-style workflow: instantiate the logistic regression model, train with fit on X and y using gradient descent, and generate predictions and scores.

Explore multiclass classification with one versus rest, turning each class into a binary problem and training per-class models; compare with one versus one, using logistic regression or SVM.

Explore one versus one multiclass classification by training a classifier for each class pair and using majority voting to finalize labels. For n classes, you generate n choose 2 classifiers.

Learn how to apply principal component analysis by mean-centering and standardizing data, computing the covariance matrix, performing eigenvalue decomposition, and projecting onto top eigenvectors to reduce dimensions.

Explain parametric versus non-parametric models, with fixed parameters in linear, logistic, and neural networks, and data-driven trees and nearest neighbors for classification and regression, noting interpretability concerns.

Apply the k nearest neighbors algorithm by computing distances to training points, selecting the k closest, and predicting by majority label for classification or averaging for regression, a nonparametric approach.

Prepare data for k-nearest neighbors by creating a two-feature toy dataset with make blobs, visualize classes, and compute distances to identify nearest neighbors for a test point.

Implement the k-nearest neighbors algorithm by computing euclidean distances from a test point to all training points, collecting labels, and predicting via the top k majority vote.

Compare Euclidean and Manhattan distance as distance metrics for data points, and experiment to see which yields better results for a given dataset.

Scale data for knn to prevent magnitude from dominating euclidean distance, and standardize features to zero mean and unit variance to normalize distances.

Explain how the cannon method handles non linear data without assuming f(x), is non parametric and instance based, and supports both classification and regression, while offering fast training.

Detect faces in images with OpenCV's cascade classifier and draw bounding boxes around them. Understand template matching, sliding windows, and multi-scale templates, plus the speed-accuracy trade-off of cascade methods.

Learn to build an OpenCV cascade classifier to detect frontal faces in images using a pre-trained XML template, tune scale factor and minNeighbors, and draw bounding boxes.

Collect data from two people via webcam, capture 20 pictures each, crop faces, and train a supervised classifier to recognize who is in the image, addressing location invariance.

Learn to collect face data by capturing 20 images per person, organize into data folders, crop with offset, resize to 100x100, and save as numpy arrays for training.

Load and stack face data from numpy files, build a 43-sample feature matrix with corresponding labels, and map zero to Prateek and one to Moet for k-nearest neighbors classification.

Learn to perform on-the-fly face predictions with a knn model using preloaded training data, capturing webcam images, detecting faces, and displaying predicted names.

Explore how the k-means algorithm performs clustering by initializing centroids, assigning points to the nearest centroid, updating centroids to the mean of their cluster, and iterating until convergence.

Explore data preparation for k means by generating a 2d dataset with 500 samples using make blobs, visualizing the x features, and performing unsupervised hard clustering without labels.

This lecture covers initializing k centers for k-means, using random centers within data bounds, normalizing features, and mapping centroids to coordinates, colors, and assigned points with a pizza shop example.

Assign each data point to the nearest centroid by Euclidean distance, building per-cluster point lists and illustrating hard clustering with cluster IDs.

Update each centroid by the mean of its assigned points, then clear the cluster lists and repeat for all k clusters, only if the cluster has points.

Visualize k-means results by plotting clusters and centroids with colors, showcasing point assignments and cluster means across iterations and initialization options.