Machine Learning Algorithms and Applications

Learn how maximum likelihood estimation (MLE) and log-likelihood drive the least-squares fitting of linear and polynomial regression, tune weights, and assess accuracy with RSS and MSE.

Explore robust linear regression using least absolute deviation with L1 norm to mitigate outliers, and ridge regression with L2 regularization to curb overfitting, multicollinearity, and hyperparameter tuning.

Explore how bayesian linear regression uses a prior and Bayes theorem to estimate parameters under uncertainty, handling limited data with a normal distribution for mean and variance.

Explore linear models for classification, including binary and multi-class setups, decision boundaries, and one-vs-k coding, then apply discriminant functions and Bayes-based probability methods to map inputs to classes.

Explore discriminant functions for classification, including binary and multi-class linear discriminant approaches, thresholding, and Fisher's criterion to maximize between-class variance while minimizing within-class variance.

Explains probabilistic generative models for creating synthetic data when data is scarce, using sigmoid and logit, Bayes theorem, and softmax for binary and multi-class classification.

Explore probabilistic discriminative models that map non-linear inputs to linear space using basis functions and logistic or probit regression, optimize weights with gradients and hessians, and apply softmax for classification.

Apply Laplace approximation to turn p(z) into a gaussian q(z) centered at the mode, and use bayesian logistic regression to estimate weights with a posterior and sigmoid classification.

Kernel functions transform nonlinear data into a linear format to aid classification, measuring similarities with rbf, linear, and string kernels, and using bag of words and cosine similarity.

Explore kernel functions from pyramid match kernels to probabilistic generative models, and use kernel tricks to compare data via histograms.

Explore support vector machines with kernel tricks, ridge and logistic losses for classification and regression, using Lagrange multipliers to identify support vectors and probabilistic outputs via sigmoid or softmax.

Explore unsupervised learning with k-means clustering, assigning points to k clusters by minimizing the distortion measure to centroids and performing e-step and m-step iterations, with alternatives like k-medoids.

Investigate how Gaussian mixture models derive latent variables from observed data to perform unsupervised clustering and assign data points to clusters via Bayes-derived posterior probabilities.

Explore an alternative view of the expectation–maximization algorithm for incomplete data, comparing hard and soft assignments in k-means and Gaussian mixtures, and applying EM to Bernoulli and Bayesian linear regression.

Explore factor analysis as an unsupervised probabilistic method deriving latent factors from observed variables, comparing exploratory and confirmatory approaches, and using correlation, eigenvalues, scree plots, and rotations.

Apply principal component analysis to reduce dimensionality by mapping high-dimensional data to a lower space, maximizing variance through standardization, covariance, and top eigenvectors for transformed data.

Describe how to select the number of latent dimensions in PCA using reconstruction error and scree plots, and summarize probabilistic PCA and PCA for categorical and multi-view data.

Explore unsupervised clustering by grouping data points using similarity-based and feature-based methods, with partitioning and hierarchical approaches, and learn distance measures and encoding for ordinal and categorical variables.

Explore hierarchical clustering, including agglomerative bottom-up merges guided by dissimilarity and dendrograms, divisive top-down splits with single, complete, and average link methods.

Assess clustering performance by calculating purity and rand index to determine how well data points are grouped, using true positives, true negatives, false positives, and false negatives to gauge accuracy.

Explore sequential data and Markov models, from first and higher order chains to autoregressive and Gaussian approaches, and apply them to language modeling, sentence completion, and data compression.

Explore hidden Markov models, distinguish them from Markov models, and learn how latent variables, lattice diagrams, and emission and transition probabilities support speech and activity recognition via the EM algorithm.