A comprehensive course in Logistic and Linear Regression.

Recap essential linear algebra concepts through a bank robbery chase, solving linear equations for unknowns like distance and time. Relate speeds, head starts, and unit conversions to linear models.

Learn how linear algebra underpins probability, statistics, calculus, and optimization, and apply it to machine learning, deep learning, and regression, including PCA, SVD, and dimensionality reduction.

Explore vectors and scalars, from 1d to multi-dimensional spaces, learn how magnitude and direction define vectors, and why linear equations stay solvable beyond human visualization.

Define a point by coordinates in increasing dimensions, generalize axes as x1, x2, ..., xn, and compute distances from the origin or between points using the extended pythagorean formula.

Define vectors as quantities with magnitude and direction, unlike scalars which have magnitude only, represented by a point on the x–y plane from the origin.

Explore row and column vectors, their shapes, and how compatible dimensions govern vector multiplication and dot products, with implications for linear and logistic regression.

Understand the transpose of a matrix (A^T), where rows become columns and columns become rows, reversing the shape from m by n to n by m.

Explore how to compute a vector's magnitude as the l2 norm, understand Euclidean distance, and obtain unit vectors by dividing by the magnitude, with 2D and 3D examples.

Explore vector operations, including element-by-element addition and the dot product, with attention to equal lengths and matching row or column formats. Visualize results and generalize to n dimensions.

Define the inverse of a vector as having the same magnitude but opposite direction. For example, (2,3) and (-2,-3) sum to the zero vector with magnitude sqrt(13).

Explore the dot product of two vectors and its compatibility criteria, contrasted with the cross product. See how the dot product yields a scalar in data analysis and machine learning.

Multiply a vector by a scalar by scaling each component, yielding a vector in the same direction with magnitude scaled by the scalar. A negative scalar reverses direction.

Explore the distributive properties of vectors and how scalars distribute over vector addition. Understand the angle between two vectors, theta, and why the smallest anti-clockwise angle is used.

Compute the angle between two vectors using the dot product and magnitudes, via cos theta. If the dot product is zero, the vectors are perpendicular, giving theta at 90 degrees.

Explore orthogonal vectors as perpendicular directions with a dot product of zero, illustrating 90-degree angles and how magnitudes and cos theta relate to orthogonality through examples.

Learn orthonormal vectors, orthogonal vectors with unit magnitude, and convert any vector to a unit vector by dividing by its magnitude. Relate dot products to magnitudes and angles.

Explore the equation of a line as foundation for linear models; express it as y equals mx plus c and as dot product form w dot x plus w naught.

Learn how a line is expressed as w^T x plus w0, where w0 indicates origin crossing. Extend this to n dimensions, linking the equation to linear algebra and machine learning.

Relate the line x minus y equals zero to y equals x, a 45-degree slope, and show that w^T x = 0 encodes a vector perpendicular to the line.

Demonstrate how lines through the origin satisfy w^T x = 0 with w as the normal vector, and how non-origin lines follow w^T x + w0 = 0.

Explore how to project a vector onto a line in the plane using coordinates and angles, deriving projections on axes and arbitrary lines.

Learn to compute the perpendicular distance from a point to a line using the vector form w^T x + w0 = 0 and projection, with origin and offset cases.

Explore how the line equation w^T x + w0 marks the side via the sign of w^T P + w0 over ||w||, and how cos theta projection defines distance.

Explore the matrix as a 2d array and its relation to vectors. Define size as m by n and distinguish row vectors, column vectors, and A11 indexing.

Master matrix operations, including element-wise addition, Hadamard product, and dot product. Understand matrix multiplication and how linear equations become matrix form in regression contexts.

Explore key matrix concepts across transpose, square, symmetric, identity, and diagonal matrices, with examples of dot products and scalar multiplication used in linear and logistic regression.

Explore orthogonal matrices, where every row and column is an orthonormal vector. Observe that the transpose product yields the identity, and see how diagonal, symmetric, and identity matrices illustrate properties.

Explore minors, cofactors, and determinants, learn to compute 2x2 and 3x3 determinants, and connect these to matrix inversion and division using the inverse of B.

Learn to identify and compute the inverse of a square matrix using determinant, transpose, cofactors, adjoint, and understand when the inverse exists and orthogonal matrices where transpose equals inverse.

Explore logistic regression from first principles, covering binary and multi-class classification, model interpretation, data preparation, and simple code, with three understanding methods: geometric, probabilistic, and log loss.

Explore the geometric method for binary classification in logistic regression by representing classes as plus one and minus one, and find a separating hyperplane that minimizes misclassifications while maximizing distances.

Formulate an optimization problem to maximize the signed distance y_i w^T x_i, ensuring correctly directed points lie far from the line. Minimize misclassifications by selecting a plane with weight w.

Examine how outliers bias the logistic regression optimization and misclassify points. Highlight squashing the distance w^T x to bound between 0 and 1, limiting outlier impact.

After obtaining the logistic regression equation, compute w^T x for a new point, apply the sigmoid, and interpret the probability with 0.5 indicating the decision boundary.

See how the sigmoid output stays between 0 and 1, and how adding a log does not change the logistic regression optimization, tying probabilistic and log-loss interpretations together.

Transform a maximization problem into minimizing log loss in logistic regression, by using the sigmoid probability of belonging to the positive class and summing the log loss across data points.

Explore how regularization balances log loss and margin in logistic and linear regression, using lambda to tune the trade-off between misclassification reduction and margin maximization.

Classify a point in logistic regression by the sign of w^T x, assigning +1 if positive and -1 if negative. Alternatively, estimate probability with squashing function and classify at 0.5.

Investigate bias and variance in modeling. Learn how high variance causes overfitting on training data and how high bias leads to underfitting, with regularization and cross-validation in logistic regression.

Explore the difference between L1 and L2 regularization, how L2 relates to the sum of squared weights, and how to choose using a validation dataset to optimize model accuracy.

Explain that logistic regression yields a linear decision surface: a line in 2D or a plane in higher dimensions, contrasted with non-linear surfaces from methods like KNN and decision trees.

Elastic net combines L1 and L2 regularization by adding both penalties to the loss function, and it requires tuning two hyperparameters, lambda and lambda1, increasing experimental complexity.

Explore explainability and feature importance in logistic regression, showing how weight magnitudes identify top features that drive positive or negative predictions and boost model interpretability.

Explore how an unbalanced dataset affects logistic regression, why discarding data is a limited balancing approach, and how squashing and balancing influence the separating hyperplane and misclassification.

Standardize the data before logistic regression to align features across units, and apply min-max or mu-sigma standardization to convert magnitudes into proportions.

Explore time and space complexity in logistic regression, covering train and test phases, resource constraints, and how memory, CPU, and data size influence performance.

Understand that a similarity matrix encodes distances between points and has a zero diagonal. Realize logistic regression uses distance to a separating plane and will not work with similarity matrix.

In high-dimensional spaces, storage and training budgets grow, and test-time computation increases. Yet higher dimensions offer greater flexibility to fit multiple planes, increasing the chance of discovering the optimal hyperplane.

Explore multiclass classification with one versus rest extensions of logistic regression, comparing three binary models and using class probabilities to predict the highest-scoring class.

Explore the probabilistic and geometric interpretations of logistic regression, showing they converge in two-class cases and clarifying how y labels differ between the methods and their probability expressions.

Explore how logistic regression's log loss interprets classification confidence from geometric, probabilistic, and loss perspectives, comparing zero-one loss and convex, continuous logistic loss to misclassification and distance from the hyperplane.

Explore the DonorsChoose dataset through practical data analysis, learning to extract insights and tell an honest story while preparing to predict proposal approval in a Kaggle competition.

Explore Kaggle DonorsChoose data by inspecting train and resource csv files, map project ids to resources, and analyze fields like id, teacher_id, school_state, project date, and category details.

Develop a data dictionary and clearly define each column, analyze data definitions, and map project IDs, titles, grade categories, categories, subcategories, resources, and submission dates for data-driven project approvals.

Connect Colab to Google Drive, load train and resource data, inspect shapes and missing values, clean and merge essays, and note class imbalance for future regression modeling.

Analyze univariate data by grouping applications by state to compute the percentage approved and rank states by approval rates. Examine how teacher prefixes and grade categories affect submissions and approvals.

Explore univariate analysis of project subcategories by cleaning and normalizing into underscore-separated keys, and count category mentions to reveal top proposals like literacy_language and history_civics.

Explore univariate analysis of project sub categories and titles, including cleaning, counting word lengths, and comparing approved versus unapproved proposals with box plots and pdfs.

Perform feature engineering by concatenating essays and counting words, then compare univariate distributions of total price between approved and non approved projects using PDFs.

Conduct univariate analysis on the project resource summary, examining length and digit presence to show how numeric digits correlate with approval rates, using crude digit detection and simple visualizations.

Discover how bag of words converts text into numerical features for modeling, using a corpus and a word mapping to create binary word indicators for logistic regression.

Analyze term frequency as a numeric text representation, contrasting it with bag of words, and learn to normalize by sentence length to reduce bias and produce comparable counts.

Explore how term frequency and inverse document frequency create tf-idf, identify topic-specific words, and filter stop words across a corpus to emphasize meaningful terms.

Learn how word2vec represents words as vectors and how sentence vectors emerge from averaging word vectors. Compare this with bag-of-words, tf-idf, and sparse matrices to understand computational trade-offs.

Learn text processing for NLP, including essay concatenation and regex-based cleaning with custom stopword lists. Build data-cleaning functions and apply them to essays and titles to produce a clean frame.

Explore how to split data into train, test, and validation sets for logistic regression, using stratified sampling to preserve class balance and avoid data leakage.

Vectorize categorical data by converting each category into a 51-state one-hot vector. Turn text fields into numerical features for regression models, and consider trade-offs between simple and complex models.

Learn to vectorize categorical data with sklearn's count vectorizer by supplying a vocabulary or deriving it from training data, producing one-hot encoded features and expanding to many categories.

Vectorize text data with a count vectorizer, controlling the corpus with minimum document frequency, max features, and n-gram settings to create a bag of words.

Explore tf-idf vectorization built on bag of words, combining term frequency with idf, and compare tf-idf vectorizer versus tf-idf transformer with count vectorizer.

Explore how to build and use word2vec representations for text data by loading glove vectors, computing average and tf-idf weighted word2vecs, and applying them to train, cross-validation, and test data.

Learn how linear regression predicts continuous values, such as house prices or stock prices, and distinguish it from logistic regression, a classification technique.

Explore linear regression intuition with a graphical age-height example, using age as input and height as output, a linear prediction line, and a loss minimized by weights and bias.

Understand mean squared error as a loss function for linear regression, measuring the average squared difference between actual and predicted values, while logistic regression uses log loss.

Compare the mean squared error of predictions to identify the best linear regression line and select the line with the lowest loss.

Explain the R square metric for linear regression, its formula 1 minus RSS over SS, and how it measures explained variance from the dumb model to the best model.

Learn how standard deviation and variance reveal data spread beyond the mean, covering population versus sample, n-1 correction, and the coefficient of variation.

Explore covariance between two variables, compare it to variance, apply the covariance formula, and interpret positive or negative relationships through correlation.

Explore how to measure associations with Pearson correlation, covariance, and scatter plots. Compare linear and monotonic relationships using Spearman rank correlation, and understand correlation versus causation in data.

Explore how R square relates to R, using RSS and TSS to compute R square from example of 0.81, and note that R square shows correlation strength but not direction.

Explain why we select mean squared error as the loss for linear models, emphasizing differentiability, continuity, and convexity, and contrast with absolute value and higher powers and their outlier effects.