A Mathematical and Programming Course on Machine Learning

Explore descriptive statistics as the foundation of machine learning, covering mean, variance, mode, and standard deviation, plus how normalization and z-scores situate data within the normal distribution.

Explore probability distributions, including Bernoulli, binomial, and Gaussian, and learn how PMF, PDF, and CDF describe discrete and continuous variables with mean mu and standard deviation sigma.

Explore the naive Bayes classifier, its conditional independence of features x1 to xM, and how log likelihood and posterior maximization drive classification.

Learn how regularization adds a penalty to the cost function to prevent overfitting by constraining weights. Explore ridge, lasso, and elastic net, and how lambda and cross validation set balance.

Explore principal component analysis to reduce high-dimensional data by projecting onto orthogonal components that maximize variance. Learn how covariance, eigenvalues, and eigenvectors guide component selection and dimensionality reduction.

Demonstrate linear discriminant analysis for dimensionality reduction, maximizing class separation via within-class and between-class scatter, and projecting data onto discriminative axes for two-class and multi-class cases.

Learn to build principal component analysis from scratch in Python by selecting eigenvectors with maximum variance, forming a covariance matrix, and projecting data for dimensionality reduction and classification.

Explore the k-nearest neighbors algorithm in Python, using Euclidean distance and majority voting to classify a test instance from nearby training samples.

Learn multiple linear regression with multiple features and data. Use least squares to estimate coefficients, assess model fit with R-squared and significance tests, and apply to a trucking time example.

Explore the significance test in multiple linear regression by using the F test for overall and individual significance, with p-values, alpha, and diagnostics like heatmaps and variance inflation factor.

Explore four decision tree approaches, including ID3, C4.5, CART, and GUIDE, and learn how entropy and information gain select root and splits while pruning and Gini impurity aid CART.

Explore the C4.5 decision tree algorithm, its improvements over ID3, and how entropy and information gain guide categorical data splits in supervised learning.

Explore the implementation of CHAID using Python to build decision trees, using entropy, information gain, and gain ratio, and compare with C4.5 while analyzing accuracy.

Explore what a recommendation system is, its real-life applications, and three major approaches: content-based, collaborative filtering, and hybrid filtering, with examples like Netflix and Amazon.

Explore content-based filtering for movie recommendations by merging genres and user comments, vectorizing text, and using SVD to build a compact similarity-based recommendation model.

Learn how support vector machines classify data with linear and nonlinear boundaries by maximizing the margin and using support vectors to define the separating plane.

Describe constrained optimization in support vector machines, maximizing margin with Lagrange multipliers, identifying support vectors via alpha, and applying kernels for non-linear classification.

Explore the implementation of support vector regression, focusing on fitting a regression line with a margin and an error tube, using kernels like a second-degree polynomial.

Explains one-vs-one and one-vs-all multiclass SVM strategies, with nonlinear separation via mapping to higher dimensions and the creation of multiple hyperplanes for each class pair.

Implement a practical perceptron classifier by wiring input features, weights, and bias into a simple neuron, and classify samples into two classes using Python.

Divide data into small batches, compute mean squared error per batch, and update weights after each batch; mini-batch gradient descent sits between stochastic and full-batch methods in neural networks.

Explore convolutional neural networks by building a sequential network with 28x28 grayscale inputs, 3x3 filters, max pooling, flattening, and a dense layer, trained with Adam and categorical cross-entropy.

Explore OpenCV basics for computer vision in Google Colab, loading images, performing color conversions (bgr to rgb), and applying translation, rotation, resizing, cropping, flipping, and dilation or erosion.

Apply bottom-up hierarchical agglomerative clustering, building successively larger clusters from Euclidean distances and dissimilarity matrices, and compare linkage types such as single linkage to reveal evolving cluster structures.

Learn to implement hierarchical agglomerative clustering with PCA to reduce high-dimensional data to two components, explore threshold-based cluster counts, and assess cluster quality with compactness and the silhouette coefficient.

Explore the fuzzy c-means algorithm by initializing memberships, computing centroids, and updating cluster memberships using inverse distance weights, enabling probabilistic boundaries across two clusters.

Explore independent Gaussian mixtures for multivariate data, derive class-conditional probabilities with the multivariate Gaussian distribution, and apply Bayesian classifiers for classification and regression.