Python, Matrices, and Linear Algebra for Data Science and ML

Explore vectors, matrices, and tensors through geometrical and numerical approaches, connecting linear algebra to data science, machine learning, and AI with applications in graphics, optimization, and analytics.

Compare scalars and vectors using a ball example, define displacement as a vector with magnitude and direction, and describe geometric and numerical representations as vectors or arrays in two-dimensional space.

Explore the numerical perspective of vectors as ordered lists of numbers, using house features like bedrooms and bathrooms to show how order matters and coordinates relate to vectors.

Explore how vector addition combines two vectors geometrically and numerically to form a resultant, and examine scalar multiplication that scales vectors without changing direction.

Explore how matrices and tensors capture geometric transformations, including square, symmetric, and triangular matrices, diagonal and identity forms, and extend to tensors for image data, enabling vectorization in machine learning.

Explore the concept of linear transformation as a function, linking input-output mappings to vectors and matrices, and preview matrix multiplication and determinants with geometric intuition.

The lecture presents linear transformation as a matrix based function that maps input vector to output vector, emphasizing movement, geometry, and origin fixed via unit vectors I and J.

Explore matrix multiplication as composing linear transformations to form a single composite transformation, illustrated with 2x2 matrices and a 90-degree rotation, linking numerical and geometric intuition.

Explore determinant as the factor by which a transformation scales area, and understand its geometric meaning, sign for orientation, and implications of a zero determinant in two-dimensional and three-dimensional.

Explore vector operations in Python and matrices, including dot products, cross products, matrix addition and transpose, solving linear equations with inversion, and matrix rank.

Explore matrix operations by adding matrices of the same dimensions, transposing to swap rows and columns, and computing the trace and Frobenius norm, with practical examples.

Represent a set of linear equations using a coefficient matrix, a variable vector, and a constants vector, then solve with inversion to get x = M^{-1} v.

Learn to invert a matrix numerically using determinant and adjugate, via minors and cofactors, then transpose to form the adjoint for solving MX = V.

Compute the inverse of a matrix using determinant and adjoint steps (minor, cofactor, transpose), then multiply by the vector to solve x, y, z; note zero determinant prevents inversion.

Learn how the rank of a matrix equals the minimum dimensions after transformation and how third-order and second-order minors of a 3x3 matrix determine full or reduced rank.

Explore eigenvectors and eigenvalues through geometric intuition of linear transformations and matrices, showing how some vectors stay on their span while others stretch.

Numerically compute eigenvalues and eigenvectors from a given transformation matrix by solving det(M − λI) = 0, the characteristic equation, then derive eigenvectors from (M − λI) v = 0.

Derive eigenvectors from eigenvalues by solving (transformation matrix - λI)x = 0, substituting λ values, and selecting simple vectors; discuss eigen spaces, spectrum, and spectral radius.

Apply Cramer's rule to solve systems of linear equations using determinants of the transformation matrix, replacing columns to compute x, y, and z, with numerical implementation and geometric intuition.

Immerse in the geometric intuition of cramer's rule by solving Ax = v as finding x that maps to v under a transformation, using 2d parallelograms and determinants.

Explore geometric intuition for Cramer's rule by relating parallelogram area before and after a linear transformation to the determinant of the transformation matrix, solving for x and y.

Orthogonal matrices are square matrices whose transpose equals their inverse, yielding identity, preserving the dot product and unit vectors, acting like rotation matrices.

Discover sparse matrices and efficient storage with structures like dictionary of keys, lists of lists, and compressed sparse row, and apply L1, L2, and infinity norms in ML.

Explore singular value decomposition (SVD) and principal component analysis (PCR) as dimensionality reduction techniques that project data from higher to lower dimensions, reducing features and aiding learning and classification.

Singular value decomposition factorizes a matrix into U, Σ, and V^T, enabling dimensionality reduction and serving as a basis for principal component analysis and solving non-square linear systems.

Demonstrates computing the singular value decomposition of matrix X by solving eigenvalue problems for X X^T and X^T X to obtain U, Sigma, and V^T.

Navigate numerical matrix inversion by computing determinant, minors, cofactors, and the adjugate, then apply to solve m x = v.

Apply truncated singular value decomposition to represent data with a few U, Sigma, and V^T components, achieving economy size and rank-one approximations for best data fit.

Explore the geometric intuition of PCA by projecting data onto the primary and secondary axes to maximize variance, minimize covariance, and enable dimensionality reduction.

Learn how principal component analysis reduces dimensionality by mean-centering data, forming a covariance matrix, and extracting principal components via eigen decomposition or the support vector decomposition technique to capture variance.

Illustrate principal component analysis by performing a support vector decomposition of the data, derive the transformation matrix, and reduce from two dimensions to one by keeping the first component.

Learn PCA via singular value decomposition using U and Sigma to reduce to the top component. Understand how singular values quantify variance and guide component selection for thresholds.

Implement vectors and matrices in Python with NumPy in Google Colab, covering dot products and matrix multiplication. Compute determinants, inverses, and introduce eigenvalues and eigenvectors using Python.

Explore how to determine a matrix rank with numpy, distinguishing full rank and rank-deficient matrices, and perform matrix addition, transpose, and trace operations.

Learn to compute the dot product and cross product of vectors using numpy in python, then solve linear systems by forming a matrix and applying numpy.linalg.solve.

Compute dot and cross products of vectors in python with numpy, and solve linear equation systems by converting them to matrix form and using numpy.linalg.solve.

Learn to compute vector norms with numpy, including L1, L2, and infinity norms, by applying numpy's norm function to a sample vector.

Explore eigenvalues and eigenvectors in Python to solve the map coloring problem using an adjacency matrix; read data with pandas, build the matrix, and visualize coloring with four colors.

Compute eigenvalues of the 49-by-49 adjacency matrix of mainland states to bound the map coloring problem, showing 3–6 colors are possible, with four colors sufficient for the United States map.

Explore eigenvalue-based graph coloring by assigning colors to states and visualizing the results with Python plotting, showing that four colors suffice to color the U.S. map.

Learn to implement singular value decomposition and PCA in Python by building SVD from X X^T and X^T X, or using numpy.linalg.svd for dimensionality reduction.

Explore image compression using singular value decomposition, converting images to grayscale and reconstructing with multiple singular values to balance variance capture and quality.

Compute the covariance matrix c = x^T x and its eigenvectors in v. Project x onto pc space as t = x v; svd with u, sigma, v yields pca.

Explore pca on a complex dataset, applying normalization, identifying variance via eigenvectors and eigenvalues, and projecting into the principal component space with reconstruction via a rotation matrix.

Apply principal component analysis to the iris dataset to reduce four features to two dimensions for visualization. Analyze variance explained by each component to understand class separation.

Apply principal component analysis to the iris dataset, reduce four dimensions to two to capture about 85% of the variance, and visualize with three-dimensional PCA to examine class separation.