
Build a strong foundation in linear algebra and dimensionality reduction, including PCA and LDA, with practical Python examples and data science applications.
Explore why linear algebra matters for geometry, dimensionality reduction, and machine learning, and learn how vectors and matrices transform feature spaces to manage large data sets.
Explore vectors, their magnitude, direction, and geometric arrows. Learn addition, subtraction, and dot product, with length compatibility and scalar results, for row and column vectors.
Handle matrices as dimensional objects with shape m by n. Use addition, subtraction, and multiplication via dot products; scalars are 1x1 and vectors are 1x n or n x 1.
Learn how to transpose vectors and matrices, turning rows into columns and shaping non-square matrices, and explore the identity matrix I and its role in multiplying A or V.
Explore linear combinations, linear span, and basis concepts. Learn about standard basis vectors and linear independence, and see how these ideas underpin vector spaces and eigenvectors.
Explore the basis of a vector space and determine linear independence and span. Compute the determinant of a matrix and derive its inverse, illustrated with a two-by-two example.
Solve linear systems using matrix form A x = y, obtaining x by left-multiplying with A's inverse when A is invertible; use Gauss method for non-invertible cases.
Learn the Gauss method (Gaussian elimination) for solving Ax = y by forming the augmented matrix, applying row operations to create zeros, and using back substitution to find x.
Solve A x = y using gaussian elimination and explore four cases: unique solution, general solution with a parameter, scenarios with two zero rows, and no solution.
Use Gaussian elimination to determine linear independence of vectors, illustrated by a three-vector example solved with an augmented matrix to reveal the trivial solution.
Explore eigenvalues and eigenvectors and their geometric meaning under linear transformations. Learn how to find eigenpairs, a key step for dimensionality reduction.
Compute eigenvalues by solving the characteristic equation det(A - lambda I) = 0 for a 2x2 matrix. This yields eigenvalues -1 and 3.
Apply Gaussian elimination to solve (A - lambda I) v = 0 and identify eigenvectors: for lambda 3, vector [1,1]; for lambda -1, vector [-1,1].
Reduce high dimensional data by feature selection and feature extraction while preserving predictive power. Compare these methods and explain dimensionality reduction and the curse of dimensionality.
Explore the curse of dimensionality, where high-dimensional data causes sparsity and degraded model performance. Use dimensionality reduction, notably principal component analysis (PCA/PCR), to close the feature space and simplify learning.
Explore principal component analysis (PCA) for unsupervised dimensionality reduction by projecting data onto uncorrelated principal components that maximize explained variance.
Apply PCA step by step on a California real estate dataset, standardizing data, creating dummy variables, and selecting four principal components that explain over 80 percent of the variance.
Explore the covariance matrix and its diagonal variances, then compute eigenvectors and eigenvalues to obtain principal components and their explained variance.
Run four-component PCA on standardized data in Jupyter and interpret the resulting components via a heatmap of correlations, revealing PC1 as property features and PC3 seasonality.
Compute the overall mean and class means for a labeled two-feature dataset, using blue and red classes to form mean vectors and relate these to dimensionality reduction.
Explore linear discriminant analysis (LDA), a supervised dimensionality reduction method that uses class labels to create linear discriminants, maximizing between-class separation and minimizing within-class variance.
Apply linear discriminant analysis to compute between-class and within-class scatter matrices, derive eigenpairs from inverse within-class scatter times between-class scatter, and project data onto discriminants to maximize class separation.
Explore applying LDA to a wine quality dataset, loading data, standardizing features, and computing class means to prepare the between and within-class scatter matrices for LDA.
Compute the within-class scatter matrix and the between-class scatter matrix in Python using NumPy, to perform feature selection and reduce dimensionality with lda-based eigenvectors.
Compute eigenvalues and eigenvectors of sw inverse sb, sort by descending eigenvalues to form lda axes, and adjust negative eigenvalues by using their absolute values for percent variance explained.
Perform dimensionality reduction by projecting an 11-feature dataset onto two discriminants formed from the top eigenvectors with the highest eigenvalues, revealing a two-dimensional representation.
Compare lda and pca for dimensionality reduction on the wine dataset, visualize class separation, train and test classifiers, and evaluate accuracy with a confusion matrix.
Explore how lda and pca serve as preprocessing steps for a wind dataset classifier from the SBC module, with careful train/test split and standardization to avoid data leakage.
Implement and compare LDA and PCA pipelines by fitting and transforming training data, projecting test data, and timing training and prediction to assess classifier accuracy.
Evaluate the training and testing times of LDA and PCA classifiers on wine data and compare their accuracy using confusion matrices.
Do you want to learn linear algebra?
You have come to the right place!
First and foremost, we want to congratulate you because you have realized the importance of obtaining this skill. Whether you want to pursue a career in data science, AI engineering, machine learning, data analysis, software engineering, or statistics, you will need to know how to apply linear algebra.
This course will allow you to become a professional who understands the math on which algorithms are built, rather than someone who applies them blindly without knowing what happens behind the scenes.
But let’s answer a pressing question you probably have at this point:
“What can I expect from this course and how it will help my professional development?”
In brief, we will provide you with the theoretical and practical foundations for two fundamental parts of data science and statistical analysis – linear algebra and dimensionality reduction.
Linear algebra is often overlooked in data science and AI courses, despite being of paramount importance. Most instructors tend to focus on the practical application of specific frameworks rather than starting with the fundamentals, which leaves you with knowledge gaps and a lack of full understanding. In this course, we give you an opportunity to build a strong foundation that would allow you to grasp complex ML and AI topics.
The course starts by introducing basic algebra notions such as vectors, matrices, identity matrices, the linear span of vectors, and more. We’ll use them to solve practical linear equations, determine linear independence of a random set of vectors, and calculate eigenvectors and eigenvalues, all preparing you for the second part of our learning journey - dimensionality reduction.
The concept of dimensionality reduction is crucial in data science, statistical analysis, and machine learning. This isn’t surprising, as the ability to determine the important features in a dataset is essential - especially in today’s data-driven age when one must be able to work with very large datasets.
Imagine you have hundreds or even thousands of attributes in your data. Working with such complex information could lead to a variety of problems – slow training time, the possibility of multicollinearity, the curse of dimensionality, or even overfitting the training data.
Dimensionality reduction can help you avoid all these issues, by selecting the parts of the data which actually carry important information and disregarding the less impactful ones.
In this course, we’ll discuss two staple techniques for dimensionality reduction – Principal Components Analysis (PCA), and Linear Discriminant Analysis (LDA). These methods transform the data you work with and create new features that carry most of the variance related to a given dataset. First, you will learn the theory behind PCA and LDA. Then, going through two complete examples in Python, you will see how data transformation occurs in practice. For this purpose, you will get one step-by-step application of PCA and one of LDA. Finally, we will compare the two algorithms in terms of speed and accuracy.
We’ve put a lot of effort into making this course the perfect foundational training for anyone who wants to become a data analyst, data scientist, machine learning engineer, or AI engineer.