A deep understanding of deep learning (with Python intro)

Learn how artificial neural networks transform inputs into outputs using simple math, weights, biases, and non-linear activations. See how backpropagation tunes these bricks into powerful deep learning models.

Apply an experimental science approach to deep learning by running parametric experiments. Vary learning rate and epochs, measure model accuracy, and assess feasibility and generalizability.

Explore the math prerequisites of deep learning and implement them in NumPy and PyTorch, with practical Python examples in linear algebra, statistics, and back-propagation.

Explore the dot product, the core operation in deep learning, defined for vectors or matrices of the same shape, and implemented with NumPy and PyTorch in Python.

Explain the softmax function and its role in deep learning, demonstrate implementing it in NumPy and PyTorch, and show how outputs form probabilities that sum to one.

Explore sampling variability in deep learning by comparing random samples to the population mean, illustrating how law of large numbers and random sampling improve model generalization and reduce overfitting.

Explore how seeding NumPy and PyTorch enables reproducible randomness, controlling initial weights and sharing code, while recognizing when different initializations prevent the local minimum.

Learn the product rule and chain rule for derivatives of interacting functions, with practical PyTorch examples and Python implementations to support understanding of gradient descent in deep learning.

Explore how gradient descent uses derivatives and the learning rate to minimize the loss function in deep learning, adjusting weights to reduce errors and approach the minimum.

Implement gradient descent in one dimension in Python with numpy and matplotlib, and explore how learning rate and training epochs affect convergence to the 0.5 minimum.

Convert gradient descent to gradient ascent in Python code for 2d problems by flipping the gradient sign. See how these methods direct the trajectory toward local maxima.

Explore vanishing gradients and exploding gradients, and learn why they hinder gradient descent in deep learning, with strategies like residual networks, batch normalization, and regularization to prevent them.

Google Colab's revision history lets you view and restore earlier notebook versions, compare changes, and undo modifications using the file revision history and its dual-panel view.

Explore the geometric view of neural networks and the perceptron, visualizing two-input feature space, separating hyperplanes, and how gradient descent learns weights for binary and continuous predictions.

Understand how simple regression relates to the perceptron, then implement a PyTorch regression model to predict a continuous y from x using gradient descent and mean squared error.

Vary the regression slope m from -2 to 2 in 21 steps and train a deep learning model; compare mean squared error loss and the correlation between y and yHat.

Discover how removing nonlinear activation functions from hidden layers lets a linear model solve the qwerties classification problem with near-perfect accuracy, illustrating when linear solutions beat deep learning.

multilayer linear models collapse into one layer unless a non-linear activation or pooling is inserted between layers; non-linearity is essential to preserve multiple layers in deep learning.

Master multiclass classification with a multi-output neural network on the iris dataset, using a 4–64–64–3 PyTorch architecture and softmax cross-entropy training.

Take on a code challenge to merge two notebooks and build a three-group qwerties classifier in Python, training a two-input neural net with four hidden units and softmax output.

Explore generalization and its boundaries, identifying populations like adult humans, children, and animals, and how broader boundaries affect accuracy and training, dev, test sets with cross-validation.

Partition data into training, devset, and test sets using numpy and scikit-learn, then validate three-way splits for objective evaluation and reduced overfitting.

Explore implementing L2 regularization in PyTorch using SGD weight_decay, train on the Iris dataset, and analyze how varying L2 levels affects training and test accuracy.

Explore mini-batches in deep learning with PyTorch, distinguishing batch and mini-batch concepts, and understand how batching provides regularization, speeds training, and optimizes forward and back propagation.

Compare model parameters and meta parameters in deep learning, and see how architecture, hidden layers, units per layer, and optimization choices shape model performance.

Explore batch normalization in PyTorch by comparing models with and without BN on the wine quality data, using z-score normalization and highlighting training and test accuracy.

Explore how batch normalization between the input and hidden layers affects a small neural network classifying qwertys, revealing near-perfect accuracy and proving batch normalization can help even shallow models.

Explore deep learning with python by predicting residual sugar from wine features in a code challenge. Master mean-squared error, training vs. test splits, and correlation analysis.

Explore loss functions in PyTorch, including MSELoss, BCELoss, BCEWithLogitsLoss, and CrossEntropyLoss, and learn to implement a custom loss function using forward.

Build a three-category qwerty predictor from x and y coordinates using a PyTorch neural network; explore code variants, visualization, and per-category accuracy to sharpen deep learning intuition.

Look to published models for similar data, but integrate your own knowledge to avoid overfitting. Use informed random or targeted grid searches to navigate the high-dimensional metaparameter space.