A Mathematical and Programming Course on Machine Learning

Explore descriptive statistics as the foundation of machine learning, covering mean, variance, mode, and standard deviation, plus how normalization and z-scores situate data within the normal distribution.

Explore probability distributions, including Bernoulli, binomial, and Gaussian, and learn how PMF, PDF, and CDF describe discrete and continuous variables with mean mu and standard deviation sigma.

Master the machine learning pipeline from data and feature extraction through preprocessing and dimensionality reduction to training an ML model for classification, regression, clustering, and visualization.

Explore the naive Bayes classifier, its conditional independence of features x1 to xM, and how log likelihood and posterior maximization drive classification.

Explore a from-scratch practical Naïve Bayes classifier for spam detection, computing word-level probabilities from training data, building spam and ham distributions, and predicting messages.

Learn how regularization adds a penalty to the cost function to prevent overfitting by constraining weights. Explore ridge, lasso, and elastic net, and how lambda and cross validation set balance.

Explore regularization theory and practical implementation of ridge and lasso regression, using housing data to apply weight penalties with lambda and norm concepts to prevent overfitting.

Explore principal component analysis to reduce high-dimensional data by projecting onto orthogonal components that maximize variance. Learn how covariance, eigenvalues, and eigenvectors guide component selection and dimensionality reduction.

Demonstrate linear discriminant analysis for dimensionality reduction, maximizing class separation via within-class and between-class scatter, and projecting data onto discriminative axes for two-class and multi-class cases.

Learn to build principal component analysis from scratch in Python by selecting eigenvectors with maximum variance, forming a covariance matrix, and projecting data for dimensionality reduction and classification.

Explore the k-nearest neighbors algorithm in Python, using Euclidean distance and majority voting to classify a test instance from nearby training samples.

Explain linear regression with a line of best fit, minimize squared error via gradient descent on beta0 and beta1, and introduce logistic regression with a sigmoid probability threshold for classification.

Explore ordinary least squares and linear regression on synthetic data, using scatterplots, and assess significance with F statistics and p-values for the independent and dependent variables.

Learn multiple linear regression with multiple features and data. Use least squares to estimate coefficients, assess model fit with R-squared and significance tests, and apply to a trucking time example.

Explore the significance test in multiple linear regression by using the F test for overall and individual significance, with p-values, alpha, and diagnostics like heatmaps and variance inflation factor.

Explore logistic regression for binary classification, using the sigmoid function to estimate loan and credit card approval probabilities and select significant features with chi-square testing.

Explore four decision tree approaches, including ID3, C4.5, CART, and GUIDE, and learn how entropy and information gain select root and splits while pruning and Gini impurity aid CART.

Investigate entropy and information gain in decision trees, and analyze overfitting, bias, and variance to understand model generalization and the role of pruning and regularization.

Explore the C4.5 decision tree algorithm, its improvements over ID3, and how entropy and information gain guide categorical data splits in supervised learning.

Explore the implementation of CHAID using Python to build decision trees, using entropy, information gain, and gain ratio, and compare with C4.5 while analyzing accuracy.

Explore what a recommendation system is, its real-life applications, and three major approaches: content-based, collaborative filtering, and hybrid filtering, with examples like Netflix and Amazon.

Explore matrix decomposition for collaborative filtering by factorizing the rating matrix into two components, enabling outer-product predictions with user and item features, via singular value decomposition and regularization.

Explore content-based filtering for movie recommendations by merging genres and user comments, vectorizing text, and using SVD to build a compact similarity-based recommendation model.

Explore collaborative filtering for movie recommendations, contrasting it with content-based systems. Learn how user-based and item-based approaches use cosine similarity to predict ratings from sparse data and feedback.

Explore clustering methods for recommendation systems, using sentiment-based features, elbow method to choose cluster count, and unsupervised learning to reveal patterns in user ratings and item data.

Implement gradient descent to train a recommendation system with collaborative filtering. Minimize mean square error on predicted ratings and tune with learning rate using training and testing data.

Learn how support vector machines classify data with linear and nonlinear boundaries by maximizing the margin and using support vectors to define the separating plane.

Describe constrained optimization in support vector machines, maximizing margin with Lagrange multipliers, identifying support vectors via alpha, and applying kernels for non-linear classification.

Explore the dual formulation of support vector machines, showing how maximizing the margin via support vectors leads to better generalization, with kernel mappings to higher dimensions.

Learn to implement linear and non-linear support vector classifiers in python, focusing on data scaling, feature engineering, and kernel trick with polynomial kernels to improve accuracy.

Explore the implementation of support vector regression, focusing on fitting a regression line with a margin and an error tube, using kernels like a second-degree polynomial.

Explains one-vs-one and one-vs-all multiclass SVM strategies, with nonlinear separation via mapping to higher dimensions and the creation of multiple hyperplanes for each class pair.

Explore how TensorFlow builds neural and deep neural networks via a graph model with parallel execution, using the high-level API and layers like Sequential and Dense with sigmoid activations.

Explore data representation in tensors, from zero dimensional scalars to five dimensional video tensors, detailing vectors, matrices, color channels (bgr), and image shapes for machine learning.

Explore the perceptron by computing z as w transpose x plus bias, applying the Heaviside step activation to output binary 0/1 values, and outlining weight updates with a learning rate.

Implement a practical perceptron classifier by wiring input features, weights, and bias into a simple neuron, and classify samples into two classes using Python.

compare gradient descent with stochastic gradient descent and mini-batch approaches, showing how per-sample weight updates offer fast convergence while risking missing the global minimum and overfitting.

Divide data into small batches, compute mean squared error per batch, and update weights after each batch; mini-batch gradient descent sits between stochastic and full-batch methods in neural networks.

Master the back propagation algorithm in neural networks by learning how to propagate error backward, update synaptic weights and biases, and iterate through epochs to minimize the error.

Learn batch normalization for neural networks, applying normalization to input and feature vectors before activation, using gamma and beta with epsilon, computed across mini-batches to stabilize training and accelerate learning.

Train a TensorFlow model on Boston housing data to predict housing prices using k-fold validation. Build a two-hidden-layer network with 64 neurons, normalize with Z-score, and evaluate mean absolute error.

Explore convolutional neural networks by building a sequential network with 28x28 grayscale inputs, 3x3 filters, max pooling, flattening, and a dense layer, trained with Adam and categorical cross-entropy.

Explore OpenCV basics for computer vision in Google Colab, loading images, performing color conversions (bgr to rgb), and applying translation, rotation, resizing, cropping, flipping, and dilation or erosion.

Explore unsupervised learning and clustering, covering k-means partitioning, hierarchical clustering, Gaussian mixture models, and DBSCAN, with hands-on Python implementations and result analysis to guide algorithm choice.

Explore k-means clustering by updating centroids and assigning points to nearest centers, observing convergence. Note limitations with high-dimensional data and outliers, and consider fuzzy c-means or hierarchical approaches.

Learn to implement and interpret the silhouette coefficient to evaluate cluster quality, comparing distances within and between clusters, and apply the elbow method to create and test cluster configurations.

Apply bottom-up hierarchical agglomerative clustering, building successively larger clusters from Euclidean distances and dissimilarity matrices, and compare linkage types such as single linkage to reveal evolving cluster structures.

Learn to implement hierarchical agglomerative clustering with PCA to reduce high-dimensional data to two components, explore threshold-based cluster counts, and assess cluster quality with compactness and the silhouette coefficient.

Explore the fuzzy c-means algorithm by initializing memberships, computing centroids, and updating cluster memberships using inverse distance weights, enabling probabilistic boundaries across two clusters.

Learn how mean shift clustering forms clusters by iteratively shifting data toward high-density regions within a radius, converging as centers stabilize and the number of clusters emerges.

Learn dbscan density based clustering, which uses core points, an epsilon radius, and a minimum point count to form clusters while identifying outliers without predefining cluster numbers.

Explore independent Gaussian mixtures for multivariate data, derive class-conditional probabilities with the multivariate Gaussian distribution, and apply Bayesian classifiers for classification and regression.

Explore how random forest uses bootstrap samples and random feature subsets to build multiple decision trees, vote for classification, and handle missing data with proximity matrices on breast cancer data.