ML from Zero: Machine Learning, Math & Python from Scratch

Why the dot product is the single most useful operation in machine learning. See it as projection, as similarity, and as the one line of NumPy that powers every prediction you'll write.

Two ways of reading the same object: a matrix as a table of your dataset, and a matrix as something that stretches and rotates space. Build the design matrix X you'll use for the rest of the course.

From secant to tangent, and why "the derivative is zero at the minimum" is the fact every optimizer in machine learning is built on. No prior calculus assumed.

Extend the derivative to functions with many inputs — which is every model you'll ever train. See the gradient as an arrow pointing uphill on a 3-D surface, then read it off a contour map.

Gaussians, likelihood, and Bayes' theorem, built up visually. This is the foundation Naive Bayes rests on, and it's why "the most likely parameters" is a thing you can actually compute.

Gaussians, likelihood, and Bayes' theorem, built up visually. This is the foundation Naive Bayes rests on, and it's why "the most likely parameters" is a thing you can actually compute.

Your first machine learning model, written as ŷ = Xw + b. Learn the bias trick — folding the intercept into the weight vector — which simplifies every derivation that follows.

Mean squared error isn't handed to you as a formula here; it's derived from the question "what should 'wrong' mean?" Find out why we square the errors rather than summing them or taking absolute values.

The full derivation of w = (XᵀX)⁻¹Xᵀy, from setting the gradient to zero. Includes a worked three-point example done entirely by hand, so you can see the matrices fill in with real numbers.

Write fit and predict in pure NumPy, line by line. Includes the shape checks that catch the errors beginners spend hours debugging.

Your first complete project — load the data, explore it, train your own regressor, predict, and read the residual plot. Everything here is your code, not a library's

What each metric actually tells you and when each one misleads. Then the payoff: your coefficients side by side with scikit-learn's, matching to fifteen decimal places.

The Normal Equation is exact, so why does anyone use anything else? Because inverting a matrix is cubic — and at 50,000 features that's 10¹⁴ operations. Enter the iterative approach.

The chain rule applied term by term until the whole thing collapses to (2/n)Xᵀ(ŷ−y). Do this derivation once properly and backpropagation later stops being mysterious.

Three runs on the same cost surface at three learning rates, side by side — one diverges, one oscillates, one converges. The single hyperparameter most likely to break your training.

Three descent paths on one contour map, plus timings. Why mini-batch won, and how to choose a batch size that isn't just copied from a tutorial.

Apply gradient descent to the house-price data, then discover why it barely works until you scale the features. The cost surface goes from a long thin canyon to a circular bowl — and one learning rate suddenly fits every feature.

Push a straight line through the sigmoid and you get a probability. Understand odds, log-odds, and why this one function turns a regressor into a classifier.

Squared error on a classifier gives you a non-convex cost surface full of local minima. Watch that happen, then derive the cost function that fixes it, and see why the log penalty is so aggressive about confident mistakes.

A page of chain rule that collapses into Xᵀ(ŷ−y) — the exact same form as linear regression's gradient. One of the genuinely elegant results in machine learning.

Write the classifier in NumPy, including the overflow-safe sigmoid. Naive implementations return NaN on real data; this one won't.

Train your classifier on a real medical dataset — load, scale, fit, and plot the decision boundary. Then think hard about what a false negative costs here.

Why accuracy is a useless number on imbalanced data, and what to use instead. Sweep the decision threshold and watch the ROC curve trace itself out.

The algorithm with no training step at all. Euclidean, Manhattan and Minkowski distance compared — and why the metric you pick quietly changes your predictions.

Build the classifier with argsort and a majority vote. Short, readable, and slow — which sets up the next lecture

Replace the loop with broadcasting and measure the speedup. Then use k-fold cross-validation to pick k honestly, rather than tuning against your test set.

Three-class classification on real chemical data, with the decision boundary drawn out. See the accuracy jump once the features are scaled — distance-based models are unforgiving about that.

A tree growing node by node beside the feature space being carved into rectangles. The clearest mental model of any algorithm in this course.

Impurity measures computed by hand on a real split, with the numbers on screen. Once you've done this once, criterion='gini' stops being a magic string

Write the recursion that finds the best split and calls itself on each side, with the call stack visualised alongside the code.

Watch training accuracy hit 100% while test accuracy falls. Why an unconstrained tree always memorises, and what depth limits and pruning actually do about it.

Bootstrap sampling plus feature subsampling, and suddenly many overfitted trees become one strong model. Build the forest yourself and see the variance drop.

Real messy data with missing values — because handling those is the lesson. Train a tree and a forest, then read the feature importances to find out what actually predicted survival.

A 99%-accurate test comes back positive. What's the chance you're actually sick? The answer surprises almost everyone, and a grid of 10,000 people shows exactly why.

The independence assumption is plainly false for real data. Learn why the classifier remains excellent regardless — and what problem it solves that makes the trade-off worth it.

Word counts into probabilities, plus the two fixes that make it work on real text: Laplace smoothing for unseen words, and log-space arithmetic to stop underflow.

Build the tokenizer and the vocabulary yourself — no CountVectorizer import. Train the classifier and see which words actually flag a message as spam.

Extend Naive Bayes to continuous features by fitting a Gaussian per class, then check your accuracy against scikit-learn's side by side

What changes when there are no labels. Inertia defined and made concrete: the number k-means is quietly minimising every iteration.

Assign, update, repeat — animated to convergence. Two steps, and you understand the whole algorithm.

Write the loop yourself, then see a bad random initialisation produce nonsense clusters — and fix it with k-means++.

Two ways to pick the number of clusters when nobody tells you what it is, plus an honest look at how ambiguous the elbow often is in practice.

Why data becomes hopelessly sparse as dimensions grow, and why variance along an axis is a reasonable proxy for how much that axis is telling you.

The linear algebra behind PCA, built visually — a cloud of points with its principal axes emerging from the covariance matrix. No prior linear algebra needed.

Four steps in code: centre, covariance, eigendecomposition, project. Includes the SVD alternative and when to prefer it.

How many components do you actually need? Plot cumulative explained variance and use the 95% rule — then see why blindly following it isn't always right.

Compress handwritten digits to 5, 20 and 50 components and watch them reconstruct. Accuracy holds — and sometimes improves — because dropping low-variance components removes noise as well as size.