Deep Learning: Advanced Computer Vision (GANs, SSD, +More!)

Discover where to access the course code and data: Colab notebooks are the main format, while plain text Python files live on GitHub, and notebooks are not on GitHub.

Explore Google Colab, a cloud-based notebook platform that runs Python with GPU or TPU support, sharing via Google Drive, and preinstalled libraries for deep learning.

Learn how to upload your own data to Colab using wget, tf get_file, or direct uploads; load datasets with pandas, adjust headers and delimiters, and access files via Google Drive.

Explore the NumPy stack basics for deep learning, including NumPy arrays, tensors, Matplotlib charts, pandas for CSV files, SciPy tools, and scikit-learn basics for classification and regression.

Learn to handle 403 download errors from lazyprogrammer.me by downloading in a browser. Upload the file into Colab via the file explorer by drag-and-drop, and understand public IP blocks.

Learn how machine learning reduces to geometry, with regression and classification as core supervised tasks; visualize data points, features, and decision boundaries to separate categories.

Explore binary classification theory and implement logistic regression in TensorFlow 2.0 using Keras, building a Dense layer with sigmoid activation and training via binary cross-entropy with validation.

Beginner-friendly guidance on how to approach coding lectures, explaining why coding along hinders learning, emphasizing thinking over typing, and pointing to self-guided study resources and the course appendix.

Perform linear classification on the breast cancer dataset to predict malignant versus benign tumors using TensorFlow 2 with a sigmoid output. Train and evaluate with a train-test split and accuracy.

Load and normalize data, build a no-activation linear regression model, train with SGD using mean squared error, and evaluate; apply log-transformed regression to Moore's Law.

Learn regression in TensorFlow by building a two-layer model and visualizing loss with Matplotlib. Apply a log transform to analyze exponential growth and estimate doubling time.

Explore how linear and logistic regression form the foundations of neural computation, showing how weighted inputs, bias, and the sigmoid produce neuron-like action potentials.

Explore how a model learns from linear regression to gradient descent, using mean squared error and learning rate eta to update weights w and b with automatic differentiation in TensorFlow.

Explore making predictions with TensorFlow 2 using the Keras API, converting predicted probabilities to class labels, flattening outputs, and validating accuracy via manual checks and model.evaluate for classification and regression.

Save and load a TensorFlow model with Keras, load via tf.keras.models.load_model, verify accuracy, and avoid a bug by not using explicit input layer, specifying input shape in dense layer.

Share your background, course, and difficulty through the suggestion box at lazy programmer.me/suggestions to help tailor advanced computer vision content, address missing explanations, and request future topics.

Introduce artificial neural networks, focusing on feedforward networks, their architecture, activation functions, and multi-class classification, with applications to image data.

Explain forward propagation through a deep neural network, showing how inputs pass through layers and neurons, compute Z with weight matrices and biases, then apply a sigmoid for binary classification.

Neural networks learn non-linear boundaries with multiple neurons and hidden layers, removing the need for manual feature engineering. Gradient descent optimizes weights to fit complex data and learn non-linear boundaries.

Explore activation functions from sigmoid and tanh to ReLU, Leaky ReLU, ELU, and Softplus, and learn how they address vanishing gradients and dead neurons in deep networks.

Learn how multi-class classification maps final-layer activations to a probability distribution over K outcomes using the softmax function, with examples like ImageNet's 1000 categories.

Explore how images are represented in computers, with height, width, and RGB channels, covering quantization, grayscale, 3d tensors, and how flattening yields N x D inputs for neural networks.

Explore why mixing primary colors in paint doesn't yield white, and how RGB color representation in computers differs, illustrating basic color mixing with practical examples.

Load the MNIST handwritten digits data from tf.keras.datasets, build a feedforward Keras model with flatten, dense, and dropout layers, train with sparse_categorical_crossentropy, and evaluate its predictions.

Demonstrates a feedforward neural network for MNIST image classification using TensorFlow 2.0 in Colab, covering data normalization, model layers, training, evaluation, and confusion matrix analysis.

Demonstrate a Colab notebook that trains a two-layer neural network on synthetic two-dimensional data to fit a non-linear cosine function, visualizing the 3D surface and loss convergence.

Explore convolution as an image modifier: slide a filter over an input image to produce a transformed output via addition and multiplication; deep learning treats this as cross-correlation.

See convolution as a sliding pattern finder that uses dot products and cosine similarity to detect patterns, linking cross-correlation and pearson correlation in feature extraction.

Explore the equivalence between convolution and matrix multiplication, illustrate with a one-dimensional example, and show how weight sharing enables efficient, translationally invariant pattern finding across image locations.

Extend convolution to color images by using a 3D input and 3D filter, producing multiple feature maps via stacking outputs and applying bias and activation in a neural network layer.

Learn the standard CNN architecture: a convolutional stage with pooling followed by dense layers, with guidance on pool sizes, strides, feature maps, and techniques like global max pooling and flattening.

Explore CNN code preparation for image classification using TensorFlow's Keras functional API, loading Fashion MNIST and CIFAR-10 data, building Conv2D networks, training, evaluating, and predictions.

Explore constructing a convolutional neural network with TensorFlow 2.0 on the Fashion MNIST dataset in Colab, including data shaping for 3D input, the Keras functional API, dropout, and evaluation insights.

Explore image classification on CIFAR-10, a 10-class dataset, using a TensorFlow 2.0 CNN in Colab, detailing data loading, architecture, GPU training, and overfitting via plots.

Explore data augmentation for image models, using on-the-fly generation with Keras image data generators to expand training data through rotations, flips, brightness, and other transforms, boosting generalization.

Apply batch normalization in convolutional networks by normalizing each batch’s mean and standard deviation, then re-scale and re-shift with gamma and beta.

Build a CNN classifier for CIFAR-10 with data augmentation using the TensorFlow Keras functional API, featuring conv layers, batch norm, max pooling, dropout, and a softmax output.

Introduce the VGG network, its history and architecture, including VGG16’s layered convolution, pooling, and fully connected blocks, and its role in image classification and localization.

Explore how VGG differs from earlier networks, why pre-trained weights accelerate development, and how Keras and open-source models like ResNet and Inception enable fast prediction without retraining.

Use transfer learning with VGG as a feature transformer, remove the last layer, and train a new classifier.

Explore greedy layer-wise pre-training for transfer learning by stacking autoencoders to build a deep network, then add a logistic aggression and compare training options.

Compare two transfer learning approaches for large CNNs: pre-compute feature vectors z and train a logistic regression, or train with data augmentation inside the loop.

Explore transfer learning with data augmentation in TensorFlow using a VGG16 pretrained model, building a small classifier with augmented images and train/validation pipelines.

Apply transfer learning without data augmentation by extracting feature vectors from a pre-trained model and training a top classifier; explore vector representations for vision, NLP, and cross-modal matching.

summarize the vgg section by highlighting transfer learning, reusing the network body and attaching a new head to train faster for SSD and style transfer.

Explore the resnet architecture, its origins, and how it overcomes LeNet-like limitations, with Keras integration and an optional TensorFlow implementation, plus application to fruits and blood cells.

Explore how residual blocks with shortcut paths enable deep networks, detailing identity and conv blocks, and show a 50-layer ResNet with no extra parameters.

Apply transfer learning with ResNet in code using TensorFlow and Keras, fine-tuning a pre-trained model on a 60-class fruits dataset. Assess performance with confusion matrices, accuracy, and comparison with VGG16.

Explore the blood cell images dataset from Kaggle and learn how to train a neural network to classify blood cells, augmenting human expertise with automated vision.

Build and train a ResNet from scratch in Keras on blood cell images, without transfer learning. Implement identity and conv blocks, and evaluate with confusion matrices.

1x1 convolutions act as a shared dense layer per pixel, performing a dot product between input feature maps to produce a C2 output, enabling per-image processing in parallel.

Explore the inception module, a multi-branch cnn with 1x1, 3x3, 5x5 convolutions and pooling, side branches for multi-scale training, including GoogLeNet, Inception V3/V4, and Inception-ResNet.

Use global pooling, including global max or average pooling, to convert variable-sized feature maps into a fixed 1 by 1 by 1024 representation, enabling the same network to process inputs.

introduce residual blocks with shortcut connections to ease identity learning in deep networks, enabling ResNet to achieve deeper architectures, while also covering inception concepts like one-by-one convolutions and side branches.

Explore how CNNs move from classification to object localization and detection with SSD, a real-time single-shot detector, and contrast it with YOLO and RCNN, including sliding windows.

Learn how to extend convolutional neural networks from classification to object localization by predicting bounding boxes and presence, using normalized coordinates and a combined loss of cross-entropy and regression.

Explore the fundamentals of object detection, from locating multiple objects and their bounding boxes to real-time accuracy in self-driving systems, and examine how ssd networks handle variable object counts.

Transform a sliding window search into a convolutional network to locate objects in an image, turning dense layers into convolution and enabling single shot detection (SSD) without region proposals.

Address the problem of scale in object detection by leveraging SSD's multi-scale feature maps and attached mini networks, using transfer learning with backbones like VGG, ResNet, and Inception.

Addressing shape and aspect ratio in object detection, SSD applies default boxes at every scale (small, big, tall, wide) to decide if objects fit, combining boxes with regression.

Explore how to set up TensorFlow's separate object detection API, install protobuf, clone the models repo, and select a pre-trained SSD model from the zoo with MobileNet as the backbone.

Learn to use the SSD TensorFlow object detection API to load COCO labels, load a saved model, run detections, and visualize bounding boxes and class names on images.

Learn to run object detection on video by looping through each frame, using a pre-trained TensorFlow object detector, and compare performance when using the GPU versus not using the GPU.

Apply the Jaccard index, or intersection over union, to measure overlap between ground-truth and predicted bounding boxes. Use non-max suppression to keep a single, high-scoring box per object.

Learn how SSD enables fast, accurate object detection by unifying localization and classification, addressing scale, shape, and sliding window challenges, with IOU and non-max suppression, for applications like self-driving cars.

Explore neural style transfer by applying an artwork's style to a photo with two inputs, learn theory and coding steps to optimize and balance content and style with a CNN.

Explore the theory of neuro style transfer, balancing content and style by optimizing the input image via content loss and style loss using a pre-trained VGG CNN and gram matrices.

Optimize the input image by minimizing the combined content and style loss with L-BFGS in Scipy, using gradients from the Keras/Theano backend and wrapping the interface for vector inputs.

Recreate content in code as the first half of style transfer by building a VGG16 average-pool network, defining a content loss, and optimizing an input image.

Learn to recreate style in code using gram matrices and multi-output VGG features, and optimize style loss to transfer painting style, such as Starry Night.

Combine prior style transfer code to generate a final output by loading content and style images, building vgg-based models, and optimizing losses with gradients.

Learn neural style transfer in advanced convolutional networks by balancing content loss and style loss to produce images with content structure and style from a reference image, using gram matrices.

Learn how class activation maps turn image classification into object localization by overlaying a heat map generated from class-specific weights in a pre-trained ResNet.

Demonstrate drawing a class activation map with ResNet using class_activation_maps.py. Load images, extract 7x7x2048 features, weight them, upsample, decode predictions, and overlay CAM on the image.