Modern Deep Convolutional Neural Networks with PyTorch

Defines what an image is, explains grayscale and color channel representations, and contrasts hand-crafted image descriptors with deep learning's automatic feature learning in neural networks.

Understand why convolutional networks replace full linear layers for 3-channel 256×256 images, avoiding a neuron per pixel by using small kernels and local parameter sharing.

Compare convolutions to linear layers by sliding three-dimensional kernels over multi-channel images to produce feature maps, achieving far fewer parameters (about 300 vs 19 million) and enabling deeper networks.

Analyze convolution parameters: stride and padding. Observe how stride alters output size and how zero or mirror padding balances pixel contributions.

Compare non-linear activation functions used in modern deep convolutional networks, highlighting sigmoid and hyperbolic tangent gradient issues. Prefer ReLU and Leaky ReLU, and use validation to select the best non-linearity.

Explore max pooling and average pooling for downsampling in convolutional networks, using 2x2 windows to reduce images while preserving essential features; differences are small, and max pooling is often preferred.

Build a deep convolutional network by stacking blocks with non-linearity, reshape outputs to a vector, apply linear layers with non-linearity, and end with soft marks for classification or regression.

Build a convolutional neural network in PyTorch with kernel size 5, stride 2, padding 0, channels 16, 32, and 64, followed by a 64-32-10 classifier.

Analyze overfitting in deep networks and apply L2 regularization to constrain weights. Compare deeper networks with regularization against smaller networks without, and tune lambda on the validation set.

Explore dropout and dropconnect regularization as deep learning techniques that randomly zero features or weights during training, and understand training versus testing, Bernoulli masking, and tuning the dropout probability.

Explore dropblock regularization for deep convolutional networks, learning to zero random image blocks during training, improving classification accuracy and including block size, masking, and class activation mapping visualizations.

Learn how early stopping regularization uses a patience parameter to halt training when validation loss stops decreasing, leveraging learning curves to prevent overfitting.

Understand batch normalization, which speeds training by normalizing data to zero mean and unit variance using feature-wise statistics, epsilon, and learnable parameters, with comparisons to layer normalization.

Explore how data augmentation extends your dataset and improves neural network quality by rotating, flipping, scaling, adjusting contrast, and adding noise.

Explores existing datasets for training neural networks, including Mooney's dataset, Fashion-MNIST, CIFAR-10 and CIFAR-100, and ImageNet, and notes how adding labeled data from these sets can improve performance.

Examines modern evolutionary architectures, from Inception modules to residual networks, showing how multi-scale convolutions and 1x1 reductions with skip connections overcome training challenges on image tasks.

Explore transfer learning by using a pre-trained ImageNet model to boost small-data classification with data augmentation. Replace or fine-tune the final layers to classify airship types, leveraging early generic features.

Load the Kaggle image dataset via an image folder, apply resize to 224, to tensor, and normalize, then create train and test loaders for a small nine-class dataset.

Learn to build data augmentation pipelines in PyTorch using transforms, including resizing, random rotation, flips, and grayscale, to boost image datasets for deep CNNs.

Reuse pretrained networks like ResNet-18 with torchvision, freeze the feature layers, replace the final layer for nine classes, and train on a small dataset to reach about 70% accuracy.