College Level Neural Nets [II] - Conv Nets: Math & Practice!

Clarify the difference between convolution and cross-correlation in neural nets, showing how flipping the filter affects the operation, the learned weights, and feature detection across frameworks.

Explore why convolution flips the kernel and how it relates to cross correlation. See how time-domain convolution corresponds to multiplication in frequency domain via the Fourier transform and commutative property.

Explain the discrete convolution as a summation with a filter, show the commutative property, and contrast it with cross-correlation for 2D images used in CNNs.

Demonstrate cross correlation in conv nets using kernel weights and activations from the previous feature map. Include bias and a 2x2 example to map outputs across locations.

Derive backpropagation through pooling layers in convolutional networks, detailing how delta flows through max and average pooling with practical formulas for relaying errors to preceding layers.

Derive backpropagation rules through convolutional layers using a simple example with flipped kernels. Explain how activation derivatives and bias affect delta propagation across valid convolutions and different kernel sizes.

Derive the full convolution form of partial derivatives in backpropagation through CNNs, showing how deltas convolve with flipped weights to update activations.

Derive the gradient of the cost with respect to CNN weights in backpropagation, using convolutional kernels, deltas, and cross-correlation with flipped filters, while noting biases and batch normalization considerations.

Explore how back propagation extends to convolutional neural networks with multiple filters and channels, deriving independent update equations for each filter and channel, and enabling parallel computation on feature maps.

Explore back propagation through CNNs with multiple input and output channels, showing how to merge equations and reveal that each input channel affects only its corresponding filter channel.

Derive backpropagation rules for strides greater than one, including dilation, padding, and valid versus full convolution, and apply them to compute weight derivatives using deltas.

Demonstrate backpropagation with strides by analyzing a 5x5 input, 2x2 kernel, stride 3, showing flipped weights in convolution drive derivatives and that many inputs have zero derivatives.

Derive backpropagation for conv nets with strides, convert the rules to a convolutional form via cross-correlation, compute weight gradients for gradient descent, and generalize to multiple filters and channels.

Explore higher-level convolutional ideas by introducing 3D, depthwise, pointwise, and dilated convolutions, and networks such as inception, residual modules, and mobile architectures, with applications to segmentation, localization, and object detection.

Explore 3D convolutions, where filters span height, width, and depth, producing multiple output channels and capturing temporal or depth information for video action recognition and 3D images.

Explore dilated (atrous) convolutions to expand the receptive field without increasing parameters, by inserting zeros between kernel elements and tuning the dilation factor L to balance receptive field and computation.

Explore global pooling in convolutional neural networks, using global average pooling or global max pooling to reduce parameters before classification, while noting tradeoffs in spatial information and input size flexibility.

Pointwise convolution (1x1) learns channel-wise weights to summarize many-channel feature maps into few outputs. This enables fully convolutional networks, avoids flattening, and supports varying input sizes.

Explore spatially separable convolution: factorize a 3x3 kernel into 3x1 and 1x3 to reduce multiplications and parameters, noting decomposability limits and possible accuracy trade-offs.

Explain depthwise separable convolutions, detailing depthwise and pointwise steps to reduce multiplications and parameters while maintaining performance compared to regular convolutions.

Explore deconvolution, also called transposed or upsampling convolution, and learn how upsampling reverses downsampling to restore original image dimensions for quality enhancement, segmentation, and encoder decoder models.

Explain how to perform transposed convolution using backpropagation-inspired logic, selecting stride and filter size to produce a feature map with the desired dimensions.

Explore why deconvolutions are called transposed convolutions, via a matrix-multiplication view that links input vectors, weight matrices, and outputs, and discuss when exact reconstruction is possible using pseudo-inverse and backpropagation.

Explore the chequerboard effect in deconvolutions and how convolution with zeros creates pixelation. Learn how stride and filter size influence the effect and how resizing with interpolation can avoid it.

Explore grooved convolution, also known as group convolution, and its historical impact from AlexNet’s 2012 ImageNet breakthrough to memory constraints and data and model parallelization.

Explore grouped convolutions and model parallelization, splitting input feature maps into groups to perform independent, parallel computations, then merge outputs for final classification.

Group convolutions offer faster training and allow larger models by boosting parameter efficiency. They reduce parameter count, save memory, and promote less redundant, better feature representations across grouped inputs.

Explore how shuffled grouped convolutions mix features across groups to enhance information flow, using fixed channel shuffling to improve performance.

Pointwise group convolution merges group convolution with a one-by-one convolution, reducing parameters via multiple groups, a concept linked to shuffled group convolutions and separable convolutions.

The Inception idea combines multiple filter sizes in a single layer to capture features at different scales, applying 3x3, 5x5, and 7x7 convolutions and stacking outputs while preserving dimensions.

Explore skip connections and residual blocks in convolutional neural networks, addressing vanishing gradients and enabling dynamic depth by learning residual functions and identity mappings.

Explore the object detection case study using convolutional neural networks, focusing on engineering challenges, reading key papers, and producing bounding boxes for multiple objects and classes.

Model object localization as regression to predict a bounding box with four coordinates, using xywh or top-left–bottom-right parametrizations, and train classification and regression with a two-head network and IoU-based losses.

Contrast class-specific and class-agnostic localization, with separate regression heads per class. Learn to localize multiple objects or landmarks and integrate with object detection for robust results.

Explore mean average precision for object detection, measuring precision and recall across probability and IoU thresholds, and learn how different definitions and multiple precision–recall curves shape results.

Explore mean average precision (MAP) by computing the area under the precision-recall curve for each class and then averaging across classes, considering ground-truth and predicted boxes with thresholds.

Explore mean average precision (MAP) calculations from precision–recall curves, comparing trapezoidal and staircase interpolations and area under the curve definitions.

Explore the rcnn object detection approach using region proposals with convolutional nets and a sliding window detector, plus transfer learning and fine-tuning ideas.

Uses selective search to generate about 2000 region proposals per image, extracts cnn features for each region, and classifies them with class-specific svms.

Region-based cnn approaches show how region proposals and deep features enable high-precision object detection, contrasting sliding-window methods and enabling fine-tuning on small detection datasets.

RCNN part 4 details non-maximum suppression for each class, using shared CNN features and tiny class-specific classifiers, plus two NMS variants with threshold tuning.

Defines a three-module RCNN object detection system: region proposals from selective search, a CNN feature extractor producing a 4096-dimensional vector, and class-specific linear SVMs. Improve localization via bounding box regression.

R-CNN test time generates about 2000 region proposals, computes 4096-d CNN features, and scores each region with class-dependent SVM weights, followed by class-wise non-maximum suppression.

Explains transfer learning for RCNN: pre-train a CNN on a large image-level classification dataset without bounding boxes, then fine-tune for detection using region proposals and a new classification layer.

Learn how RCNN trains an object-specific classifier by extracting a 4096-dimensional feature vector from region proposals before the softmax and training per-class SVMs, defining positives and negatives by IOU thresholds.

Explore hard negative mining in RCNN training, contrasting softmax and SVM positives, and show bootstrapping and data mining strategies to focus on hard negatives for faster convergence.

Learn how bounding box regression refines region proposals in RCNN to boost localization accuracy, using class-specific transformations of center, width, and height learned from proposal-ground-truth pairs.

Explore bounding box regression in rcnn by building class-specific datasets that pair proposals with ground-truth boxes using IoU thresholds around 0.6, and learn a linear mapping from pool5 features.

Learn how RCNN visualizes intermediate layer activations by selecting a unit, ranking top scoring proposals by activation, and displaying inputs that maximize responses, revealing receptive fields and channel features.

Compare RCNN variants and demonstrate bounding box regression and fine tuning boost mean average precision. Learn how pool5, fc6, fc7 features with SVM achieve state-of-the-art results.

Explore a false positive error analysis in RCNN, ranking FP types by confidence and examining localization, duplicate, similar category, other, and background errors, with bounding box regression improving results.

Explore spatial pyramid pooling (spp net), a pooling strategy enabling fixed-length representations from arbitrary image sizes, boosting speed and robustness for object detection while remaining compatible with any cnn design.

Explore how spatial pyramid pooling enables CNNs to handle arbitrary input sizes with a fixed-length SBP, reducing cropping and warping distortion and speeding up detection.

Learn how spatial pyramid pooling converts arbitrary-size feature maps into fixed-length vectors by partitioning into multi-level bins and applying pooling per bin, enabling scale- and size-invariant representations for convolutional nets.

Showcase SPP-net for object detection by extracting feature maps from the entire image once and pooling over region proposals to fixed-length features, enabling orders-of-magnitude faster testing.

Compare sppnet results for object detection with sbp variants, highlighting faster runtimes, higher mean average precision, and the impact of bounding box regression and fine tuning.

Fast R-CNN introduces a single-stage training that jointly classifies object proposals and refines bounding boxes, removing SVM and multi-stage training from R-CNN and SPPNet with major speed and accuracy gains.

Fast RCNN part 2 presents an end-to-end multitask architecture using a region of interest opening layer to extract fixed-length features and dual heads for classification and bounding-box regression.

Learn efficient fast R-CNN training by exploiting feature sharing with fixed-length pooling and sampling images, enabling backpropagation without reprocessing each image, plus a multitask loss for classification and bounding-box regression.

Explore back propagation through ROI layers in Fast RCNN, deriving how ROI pooling maps a feature map to fixed-size outputs and computes gradients through max pooling and region proposals.

Show how truncated SVD speeds up Fast RCNN by compressing fully connected layers into a two-layer factorization, reducing parameters from uv to t(u+v) and accelerating detection.

Compare fast r-cnn results to cnn and SpiNET, noting mean average precision and backbone performance. Highlight training and testing time gains and truncated DVD impact with softmax versus SVM.