Supervised Learning for AI with Python and Tensorflow 2

Explore how artificial intelligence, machine learning, and supervised learning map inputs to outputs using labeled data, with examples like English to French translation, and contrast unsupervised and reinforcement learning.

Explore supervised learning with parametric models that fix the number of parameters, and learn to initialize and iteratively update weights from training data, using a threshold function to produce outputs.

Train a simple parametric supervised learning model by formatting training data, fitting weights, and iterating to monitor progress toward convergence.

Explore a binary linear classifier by visualizing y_hat = x1 w1 + x2 w2 with weights set to 1 and the boundary x1 + x2 = 0.5, then plot points.

Explore how learning rate affects training, causing updates to weights through iterations, and learn to generalize code by using a weights list and loop-based updates.

Explore vectors as generalizations of scalars, cover element-wise operations and dot product, and learn vector notation, transposition, and element indexing to speed up code via vectorization.

Vectorize the training method by replacing loops with the dot product between X and the weights, then benchmark with random data to show five to six times faster performance.

discover how a bias node shifts the decision boundary to solve problems that previously couldn't be solved, by adding an extra input and updating weights and bias.

Explore how adding a bias node shifts the decision boundary by adjusting weights and biases, through a practical lab that updates, plots, and analyzes dynamic boundaries.

Explore the perceptron algorithm from theory to hands-on lab, learning how to initialize weights and bias, apply a threshold function, adjust with learning rate, and assess generalization with datasets.

Explore non-binary inputs and feature scaling for supervised learning. A simple perceptron trained on house data uses max magnitude scaling to reduce training iterations and improve generalization.

Learn to move from toy problems to real data by splitting datasets into training, validation, and testing sets, preventing overfitting, and handling missing values.

Download real Titanic data from Kaggle, load it with pandas data frames, perform one-hot encoding, impute missing values, and prepare training and validation features for model fitting.

Train a perceptron on scaled Titanic data after imputing missing ages, evaluate training and validation accuracy, and generate Kaggle submissions predicting passenger survival.

Learn how to save and load model weights with a simple perceptron using pickle, enabling training reuse and live predictions on large datasets.

Learn training improvements for the perceptron model, including tracking best validation accuracy, saving weights, and data ordering to boost Titanic testing accuracy and Kaggle predictions.

Explore supervised learning by adapting a perceptron from classification to linear regression, adjusting activation and evaluation with mean absolute difference, and intuitively fitting a line to data.

Demonstrates why a perceptron, a linear classifier, fails on non-linearly separable data like xor, and previews neural networks while covering basics of learning, weights, threshold, and data preparation.

Explore the fundamentals of neural networks and deep learning, including perceptrons, weights, biases, and activation functions. See how hidden layers and nonlinear mappings power models like convolutional networks and transformers.

Explore logistic regression as a linear binary classifier, using the sigmoid activation to interpret outputs as probabilities and learning parameters via gradient descent with binary cross-entropy loss.

Explore derivatives as rates of change, learn to compute dy/dx and limits, and apply five rules—power, constant multiplier, sum and difference, zero, and chain rule—to minimize the error function.

Learn gradient descent, an optimization algorithm that minimizes the error function by updating weights and biases in the direction of steepest descent using a learning rate.

Explore logistic regression equations, compute model outputs with z = w^T x + b, apply the sigmoid activation, and update weights using gradient descent with binary cross-entropy loss.

Transform the perceptron into a logistic regression model using the sigmoid activation, train on the Titanic data, and boost Kaggle predictions.

Learn matrices as two-dimensional number arrays, explore their shape and transpose, and master scalar and matrix arithmetic, elementwise operations, and dot-product based matrix multiplication.

Demonstrates vectorizing logistic regression with matrix multiplication, improving speed threefold while preserving accuracy, and setting the stage for neural networks.

Compare logistic regression to neural networks by introducing hidden layers for nonlinear classification, then master forward and backpropagation and notation for weights, biases, and layers.

Apply forward propagation in a neural network by computing z values, applying sigmoid activations, and using matrix-vector notation with weights and biases to derive layer-wise outputs.

Implement forward propagation in Python by initializing layer sizes, weights, and biases with small random values, applying sigmoid activation, and producing predictions.

Explain how to update neural network parameters with backpropagation, using gradient descent and the chain rule to compute layer-by-layer derivatives for binary cross entropy loss.

Derives backpropagation equations and their vectorized form for a two-layer neural network, showing gradient descent updates for weights and biases, and generalizes to networks with any number of layers.

Implement and train a neural network by applying back propagation, including the loss function, sigmoid derivative, and gradient descent, with forward and back propagation and parameter initialization.

See how hidden layers enable neural networks to learn via backpropagation, using nonlinear activation like the sigmoid to remap inputs into hidden features. Layers stack to learn complex mappings.

Explain why zero initialization causes symmetry and prevents learning in neural networks, and show how random, small weight initialization enables updates in hidden layers, while biases need not be randomized.

Explores multitask and multiclass classification, showing how to adapt a neural network to predict multiple diseases with a vector output, using softmax and categorical cross-entropy loss.

Explore multiclass classification by applying softmax in the last layer and using categorical cross-entropy loss, then derive the gradient of the loss with respect to the last layer z values.

Explore multiclass classification with handwritten digits by implementing a neural network using categorical entropy and softmax, then compare training, validation, and testing accuracy with various hidden layers.

Explain how the vanishing gradient problem slows learning in deep networks due to the sigmoid derivative shrinking weight updates. Introduce ReLU as a practical non-linear activation to keep updates effective.

Replace the sigmoid with the relu activation to address vanishing gradient and compare no-hidden, one-hidden, and large networks, noting training gains and overfitting risks.

Learn to analyze supervised learning models using confusion matrix analysis with a multiclass digit classifier, measure accuracy, interpret misclassifications, and visualize results to identify overfitting.

Understand how larger neural networks can overfit by memorizing training data. Learn prevention techniques such as smaller models, more data, early stopping, and L1/L2 regularization plus dropout.

Explore batch and mini-batch gradient descent, showing how dataset size and batch size affect updates, convergence, and training stability versus stochastic gradient descent.

Implement 128-size batching to track training accuracy per batch and average it for iteration accuracy; compare batching with full-batch, include data shuffling and stochastic gradient descent with batch size one.

Learn how to refactor supervised learning code by extracting parameter updates into a dedicated optimizer class, implementing gradient descent, removing learning rate, and supporting multiple optimizers for clearer, extensible training.

Explore momentum as gradient descent with a weighted average of past gradients, reducing local minima risks and smoothing updates to improve neural network training and performance.

Implement momentum in gradient descent using moving averages to update parameters with a beta. Momentum affects testing accuracy, helping small models but hindering large ones, and varies by optimizer.

RMSProp uses the moving average of squared gradients to divide the derivative by the square root. This scales learning rate, speeds up flat regions, slows steep regions, with an epsilon.

Implement the RMSprop optimizer and compare it with gradient descent with momentum, updating parameters via the squared gradient running average to stabilize softmax.

Adam optimizer, a blend of momentum and RMSProp, uses bias-corrected terms to yield accurate weighted averages during early updates, with alpha, beta1, beta2, and epsilon.

Implement the Adam optimizer in the code lab, initialize the time step and momentum terms, and update parameters using corrected weighted sums of derivatives and squared derivatives.

Introduce convolutional neural networks (CNNs) and their real-world applications, from image classification and style transfer to object detection in bloodstream images; cover padding, strides, transfer learning, and visualization.

Explore convolutional neural networks with a focus on image data. Understand how images are represented as pixel matrices with channels for grayscale, red, green, blue, and the alpha transparency channel.

Explore how filters power convolutional neural networks by detecting edges and features through a convolutional operator, producing activation maps that reveal image differences across channels.

Use zero padding to keep the activation the same size as the input, padding on each side according to the kernel size to shape the activation map.

Explore strides in convolution, applying filters across an image with stride one or more, using dot products, padding options, and output size calculations.

Learn how reshaping converts vectors and tensors into matrices for convolutional networks, mapping 784-element vectors to 28×28 images and handling three-dimensional, four-dimensional, and five-dimensional shapes while preserving order.

Learn how convolutional neural networks apply convolutional layers to 28x28 grayscale inputs, learn filters during training, and flatten activations for classification; explore stride and peeling layers to reduce parameters.

Describe forward propagation in a cnn: apply 3x3 filters with padding and stride, reshape patches into a strips matrix, multiply by filter weights, add bias, apply activation, and softmax.

Implement CNN forward propagation in a classification model by initializing convolutional filters and biases, handling input dimensions, padding, strides, and transitioning to fully connected layers with correct sizing.

Initialize h0 activation for a four-dimensional batch of images, then perform padding, extract patches, reshape to strips, and compute activations through convolutional layers before connecting to dense layers.

Extract patches from images, reshape to four-dimensional tensors, and test forward propagation in a CNN with patch-based processing using Python and TensorFlow 2.

Apply backpropagation in convolutional neural networks by deriving gradients through activation maps, patches, and padded inputs using 3x3 windows and strides.

Implement backpropagation for a convolutional neural network in a lab, adapting derivatives across convolutional and fully connected layers. Reshape activation maps, handle padding and patch stitching, and validate training accuracy.

Explore pooling layers that reduce spatial dimensions in convolutional neural networks using two-by-two windows. Learn how average pooling and max pooling affect translational invariance and forward and backward propagation.

Learn to implement a pooling layer in a cnn through forward propagation, adding a pooling window parameter, performing max pooling, and tracking max indices for backpropagation in this optional lab.

Learn backpropagation through a convolutional neural network with max pooling from scratch, computing activation derivatives and patch stitching to backpropagate through the pooling layer.

Explore using TensorFlow and Keras to build and train neural networks with high-level APIs. Learn data loading, normalization, and model definitions—sequential, functional, and subclassing—plus training with fit and optimizers.

Train and evaluate a basic TensorFlow Keras model, use predict outputs, and implement sparse and categorical cross entropy while building convolutional networks with checkpoints.

Pre-process an image dataset for classification by standardizing to 256 by 256 region via center extraction, organize class folders, and plan batch loading with tf.data for memory efficiency in Colab.

Create a Tensorflow record from a large image dataset by cropping images, encoding them as bytes, and storing image and class features, then split into train, validation, and test.

Upload and load tfrecord datasets in Colab, map records to images and labels, train a CNN on a larger dataset with GPU, and assess overfitting.

Trace the history of CNNs for image classification from two-stage methods to end-to-end models like AlexNet, Inception, ResNet, and EfficientNet, highlighting data augmentation and transfer learning.

Explore AlexNet data preparation for training a CNN, including zero-centering with batch-wise mean values and data augmentation with random crops, flips, and brightness adjustments.

Define an AlexNet-style CNN in Keras using the functional API with conv layers, max pooling, dropout, and dense softmax classifiers; train with data augmentation and discuss transfer learning.

Explore transfer learning with a pre trained image model, replace the final layer for 256 classes, freeze and fine tune layers using global average pooling, dropout, and softmax.

Explore occlusion sensitivity to visualize convolutional neural networks by sliding a window across images and creating heat maps that reveal where the model detects the snail class.

Explore neural style transfer that blends a content image with a style image by optimizing content and style losses via CNN activations and Gram matrices.

Set up a neural style transfer lab by loading content and style images, using a frozen VGG19 model, and extracting multi-layer activation maps for content and style losses.

Compute gram matrices from activation maps to quantify style, extract content and style activations from content and style images, and optimize a generated image using combined content and style losses.

Learn to perform style transfer with a gradient tape training loop, activation maps, and clipping, adjusting style weight and momentum for evolving stylized images.

Explore one-shot learning with a metric-based approach that trains a distance function to distinguish face categories using embeddings, triplet loss, and a 128-d vector, enabling face verification and recognition.

Learn face verification and recognition with a pre-trained model and a facial recognition library. Compute 128-d embeddings to perform one-to-one verification and one-to-many recognition against a labeled database.

Object detection with neural networks locates multiple objects in images using bounding boxes and a yolo-style grid that predicts centers, box coordinates, and class probabilities.

Learn how object detection loss combines binary cross entropy for object presence with regression losses for center coordinates and size, using sqrt-transformations and multi-label class losses across anchors.

Train an object detection model to locate blood cells in images by parsing XML annotations, scaling bounding boxes, and using a pre-trained network.

Select three anchor boxes via k-means elbow analysis for a 19x19 grid. Create IoU-based labels, apply a custom loss, train, and save the model.

Convert 7x7x16 prediction matrices to bounding boxes with anchor boxes, then evaluate using IoU, non-maximum suppression, and F1 score.

Extract predictions from the 3-D matrix to compute bounding boxes on images, scale from 600x600 input to 416x416, apply non-maximum suppression, and assign class labels with a 0.25 threshold.

Apply non-max suppression to filter overlapping bounding boxes by class and iou threshold, keeping the most confident predictions.

Learn to evaluate object detection with IOU-based metrics, precision, recall, true positives, false positives and false negatives, and compute classwise matrices and overall F1 scores across validation data.

Master convolutional neural networks through labs on data augmentation, custom loss functions, and model visualization, then apply transfer learning, early stopping, and a precision-recall metric for evaluation.

Explore sequential data and order-dependent architectures, build a from-scratch recurrent neural network for sentiment classification, and read and generate text from Romeo and Juliet using attention in GBG three.

Explore recurrent neural networks, weight sharing across time steps, and hidden states for sequential tasks like sentiment analysis and language translation using encoder-decoder architectures.

Learn forward propagation for RNNs in sentiment classification, using time-step processing, shared weight matrices, biases, one-hot encoding, activation functions, and batching.

Format text data for sentiment classification by building a vocabulary, padding sentences, and encoding labels; then implement a forward propagation neural network in Python with TensorFlow 2.

Learn to implement backpropagation for RNNs by defining binary cross-entropy loss, deriving time-step weight updates for the weight matrices, and applying shared derivatives across all timesteps.

Implement backpropagation in a neural network to predict sentences, explore vanishing gradient with longer sequences, and discuss mitigation using long short term memory.

Explore how long short-term memory networks address vanishing gradients by using a memory cell with forget, input, and output gates to preserve information across longer sequences.

Implement rnn, lstm, and gru models in TensorFlow to classify sentiment, loading basic and longer sentences, training with embeddings, and compare architectures and training strategies.

Master character-based text generation by training a small lstm model on Romeo and Juliet text in Google Colab, converting text to indices, one-hot encoding, and sampling new sequences.

Explore word embeddings that replace sparse one-hot vectors with dense representations, enabling similar words to share meaning through context in sentences, while noting they are not always interpretable.

Explore GloVe word embeddings by loading a 300-dimensional model trained on 840 billion tokens, mapping words to vectors, and using cosine similarity for semantic analogies.

Explore advanced sentiment classification using GloVe embeddings on airline reviews, map words to embeddings, create train/validation/test splits, and train a neural network achieving about 76% test accuracy.

Apply transformer-based BERT embeddings to sentiment classification, using a preprocessing and embedding layer, fine-tuning with Adam and sigmoid output to beat glove results with ~83% test accuracy.

Learn how the attention mechanism enhances encoder-decoder models by using an attention layer to compute a context vector from input hidden states for tasks like machine translation and text summarization.

Define, train, and visualize an attention model that maps diverse date formats to a standard date, using data loading, one-hot encoding, and a custom label matches metric.

Run the model in eager mode to extract attention values for a selected index, then plot a grayscale attention heatmap showing input characters the model focuses on when generating output.

Explore sequential data techniques for sentiment classification and text generation, and apply them to audio, sales forecasting, and image captioning to produce descriptive text.

Encourage learner feedback and constructive criticism to improve the course. Guide ongoing learning with resources like the deep learning book by Ian Goodfellow, GitHub projects, image captioning research, and competitions.