Computer Vision Masterclass

Explore face detection strategies in this section: Haarcascade classifiers with OpenCV, hog with Dlib, and CNN with TensorFlow, comparing results and ending with a webcam demo.

Explore how images are represented as matrices of pixels, each with RGB channels for color images. Contrast grayscale to illustrate how pixel data size differs.

Explore the cascade classifier for object detection, training with positive and negative images using Adaboost, learning Haar-like features, sliding windows, and staged classifiers to detect faces.

Learn to implement face detection with OpenCV's cascade classifier by loading and resizing an image, converting it to grayscale with cv2, and preparing it in Google Colab with Google Drive.

Detect faces with a pre-trained Haarcascade using OpenCV. Load the cascade and apply detectMultiScale on grayscale images to draw bounding boxes around detected faces.

test cascade parameters to reduce false positives in Haar cascades for face detection; adjust scale factor values (around 1.05–1.09) to balance detections and processing time.

Modify scale factor, min neighbors, and min size and max size to tune Haarcascades for face detection, reducing false positives while maximizing detections.

Detect eyes in images with a haarcascade eye classifier in OpenCV, tuning scale factor, min and max size, and neighbors to improve accuracy while processing face detections.

Explore using different pre-built classifiers to detect cars, clocks, and full body, and adjust parameters to improve detection results in this homework solution.

Explore hog histograms of oriented gradients to detect faces and objects by computing gradient directions and magnitudes, constructing histograms to compare features for recognition.

Detect faces with hog using the lib library in Google Colab, compare to Haar cascade and OpenCV, then preview CNN-based detection with bounding boxes.

Explore face detection with convolutional neural networks using Dlib’s cnn detector, load pretrained weights, and compare performance against hog and cascade classifiers, highlighting confidence scores and visualization.

Compare haarcascade, hog, and cnn face detectors on a challenging image to determine which algorithm best detects small faces. Hog outperforms haarcascade and cnn in this case.

Install and configure Anaconda and PyCharm on your computer to support Python AI work, using Google Colab for code and preparing for future object tracking.

Detect faces with a webcam using OpenCV and haarcascade, process live video frames in a PyCharm project, and tune the min size to reduce false positives.

Plan the approach to face recognition by contrasting face detection and recognition, implement LBP with OpenCV, and compare CNN-based recognizers using Dileep and TensorFlow.

Explore lbph, the local binary patterns histogram approach to facial recognition, from central pixel thresholding to histogram-based face features in OpenCV.

Implement face recognition with the OpenCV library and the LBP algorithm by loading the Yale faces dataset, training on the train set, and evaluating the classifier on test images.

Preprocesses face images for a recognition case study by loading gif images with PIL, converting to grayscale numpy arrays for OpenCV, extracting class IDs for the lbp classifier.

Train the LBPH classifier by dividing each image into eight by eight blocks to create 64 histograms per face, save them to a file, and compare new images against them.

Train and test an LBP face recognizer by loading a classifier, preprocessing test images to grayscale numpy arrays, predicting identities, and comparing predictions with expected outputs.

Evaluate the lbph face classifier by processing test images, obtaining predictions, and computing accuracy with sklearn's accuracy_score, then visualize a confusion matrix to interpret per-class performance.

Investigate lbph parameters such as radius, number of neighbors, grid x, grid y, and threshold to optimize recognition results and generate histograms per cell.

Experiment with LBPH parameters by adjusting radius, neighbors, and grid values, train the classifier, and compare accuracy to tailor results to your own images.

Detect faces and map 68 facial landmarks with a pre-trained dlib shape predictor in Colab, then compare 68-point accuracy with a faster 5-point model.

Explore how to implement face recognition with the Dlib library, using a 68-point shape predictor and a pre-trained resnet model to extract facial descriptors and train on a faces dataset.

Learn to implement face recognition with the dlib library by extracting 68 facial points, computing 128-value face descriptors with a convolutional neural network, and preparing data for matching.

Learn to compute face distances for recognition using the lip library's CNN and 128-face descriptors, enabling similarity-based classification across a dataset with numpy.

Demonstrate face detection and recognition with dlib, extracting 68 landmarks and a 128-feature descriptor, then classifying test faces using a 0.5 threshold.

Finish implementing and evaluating a dlib face recognition pipeline, tuning the distance threshold for detections. The convolutional neural networks approach achieves 100% accuracy on the dataset, outperforming the lbp method.

Build a facial recognizer classifier from a new dataset, capture faces with a webcam, and evaluate an LBP classifier against Dlib while preparing a test set.

Learn to run a webcam-based face recognition using OpenCV in PyCharm, applying a haarcascade detector and Elbe recognizer to identify Jones and Gabriel.

Explore the plan of attack for object tracking, contrast it with detection, and compare KCF and SRDCF algorithms with OpenCV to track people in videos.

Learn the differences between object tracking and object detection: trackers reuse previous object information for faster frame-to-frame prediction, while detectors run fresh per frame and may reset with detection.

Explore the basics of two tracking algorithms, kernel correlation filters and discriminative correlation filter with channel and spatial reliability, including particle filter initialization, bounding boxes, hog features, and confidence maps.

Learn to implement object tracking in videos using OpenCV's KCF tracker, selecting a region of interest, initializing the tracker, and updating the bounding box frame by frame.

Explore object tracking with the csrt algorithm, compare it to the prior method, and observe how csrt uses random Markov probabilities and hog features to follow objects across video sequences.

Compare KCF and CSR trackers through tests to track cars and a person in street videos. Evaluate bounding boxes and tracking efficiency to choose the best approach for your application.

Explore neural networks for image classification using a 300-image Homer and Bart dataset. Implement two approaches, from raw pixels to a custom OpenCV feature extractor, and compare results.

Delve into the biological fundamentals of human neural networks, from more than 100 billion neurons to how electrical signals and synapses drive brain information processing.

Explore the artificial neuron, including inputs from environment data, dendrites and the cell body, weights, the sum and activation functions, and how outputs are produced, illustrated by a salary-prediction example.

Explore how a two-input perceptron multiplies inputs x by weights w, sums them, and passes the result through a step function to predict outcomes like a salary increase.

See how a perceptron updates weights to classify the and operator dataset, starting from zero weights and using the sum, step functions, and a learning rate to reduce error.

Update the perceptron weights with a learning rate across epochs, compare zero-one outputs via the step function, and show how two-input nets classify simple patterns while xor needs multilayer networks.

Explore how multilayer neural networks extend the perceptron with a hidden layer, using weights and activation functions to solve non linear problems like XOR via feed forward.

Explore activation functions—step, sigmoid, and hyperbolic tangent—and see how their output ranges and weight changes affect perceptron decisions, illustrated by a salary increase example.

Explore how a multilayer neural network performs feedforward calculations from input to hidden layers, applying weights, sum functions, and sigmoid activation to predict outputs.

Continue the feedforward pass in a multilayer neural network by computing sums and applying activations for all input instances in the hidden layer, using weights and activation steps.

Explore feedforward neural networks, compute hidden-to-output activations using the sum and sigmoid functions, and interpret final output activations for predictions.

Explore how multilayer neural networks measure error using a simple loss function, comparing predictions to expected outputs for an xor-style dataset and adjusting weights to minimize average absolute error.

Visualize the multilayer neural network algorithm from random weight initialization to output calculation with sums and activations, error evaluation, and weight updates via gradient descent, delta, and backpropagation across epochs.

Learn how gradient descent updates neural network weights across epochs using the partial derivative and sigmoid derivative, guiding updates toward the global minimum of the cost function beyond local minima.

Compute delta output as error times sigmoid derivative to guide updates. Use backpropagation to update weights from right to left toward the global minimum.

Derive the hidden layer delta by applying the sigmoid derivative, weights, and delta outputs to determine the gradient direction for updating weights across multiple data instances.

Explore backpropagation in multilayer neural networks by updating weights through gradient descent with the delta parameter and learning rate, revealing how convergence to the global minimum depends on rate schedule.

Update the weights from the hidden to the output layer using inputs, deltas, activations, and a learning rate of 0.3; apply updates across neurons and replace old weights.

Update hidden-to-output weights using delta-based gradients, compute input times delta, set learning rate to 0.3, complete the first epoch, and restart feedforward with backpropagation across layers.

Explore how the bias unit shifts activations, learn error metrics such as mean squared error and root mean squared error, and handle multiple outputs with encoding in neural networks.

Apply the hidden layer size rule (inputs plus outputs divided by two) as a starting point and test two-layer versus deeper networks on nonlinear problems, using a credit-history example.

Explore gradient descent methods to minimize error by updating weights via partial derivatives, understand global and local minima on convex and non-convex surfaces, and compare batch, stochastic, and mini-batch approaches.

Trace the evolution of deep learning from early neural networks and SVMs to modern CNNs and RNNs, and explore encoders and GANs for image classification and NLP.

Explore how image pixels become neural network inputs for classification, including pixel representation, color channels, and how input/output layers scale with image size and ten classes.

Connect to Google Colab, import TensorFlow and supporting libraries (OpenCV, NumPy, pandas, seaborn, matplotlib), verify TensorFlow version 2.4.0, and prepare for pixel extraction in the next step.

Process character images by extracting pixels into a uniform 120 by 128 format for neural network input, using Google Drive mounting, unzipping the dataset, and labeling Homer or Bart.

Extract pixels from a set of images sorted alphabetically by reading files, skipping non-images, then convert to grayscale and resize to 128x128 using OpenCV.

convert grayscale 128 by 128 images to vectors using numpy ravel to form 16,384 input features, then label each image as Bart or Homer from the image name for training.

Convert lists to numpy arrays to form inputs x and outputs y for the neural network, then reshape vectors into image matrices of pixels for visualization and seaborn count plot.

Normalize image pixel data to the 0–1 range using minmaxscaler, speeding neural network processing and improving accuracy with dataset X, before moving to train and test set definitions.

Split the data into training and test sets with train_test_split, using 0.2 test size and a random state to evaluate the neural network on 215 training and 54 test images.

Build and train a TensorFlow tf.keras sequential neural network with dense hidden layers (8193 and 8000 neurons) and a sigmoid output for binary classification, using relu, adam, and binary cross-entropy.

Evaluate a neural network by comparing training and test accuracy, analyzing loss trends, and using thresholded predictions with metrics like accuracy score, confusion matrix, and classification report.

Save the neural network structure to network one.json and the weights to weights one.h5, then load, reconstruct, and compile with binary cross entropy, Adam, and accuracy for production.

Load a neural network, classify a single test image by reshaping and inversely transforming pixels from 0–1 to 0–255, and predict whether it is Bart or Homer.

Learn to extract color-based features from Homer and Bart images to build a feature dataset for a neural network that classifies the characters through an input vector.

Implement a feature extractor in a Google Colab workflow using OpenCV to perform feature extraction. Process Homer and Bart images to extract per-pixel colors and assign class Homer or Bart.

Explore per-pixel feature extraction with OpenCV by iterating image pixels, detecting Homer mouth brown using BGR color ranges, and normalizing features for a consistent dataset.

Explore feature extraction with OpenCV 3 by setting blue and gray color intervals and applying bottom-half image analysis to extract Homer’s pants and shoes.

Finish color-based feature extraction with OpenCV 4 by counting pixels for shirt, shorts, and sneakers using defined color intervals, and prepare to export features to a CSV.

Finish implementing the feature extractor with OpenCV 5 by compiling per-image features, adding the class label, and exporting a features.csv file for dataset creation and future neural network training.

Define train and test sets to evaluate a neural network by selecting features and class labels, then split with train_test_split (test size 0.2, random_state 1) into x_train, y_train, x_test, y_test.

Build and train a compact neural network for image classification using a tf.keras sequential model with hidden layers, relu activations, and a sigmoid output for binary classification.

Evaluate a neural network by monitoring loss and accuracy over epochs, generating X test predictions with a sigmoid threshold, and analyzing results via confusion matrices and classification reports.

Learn how to save, load, and classify a single image by saving the json model and weights, loading them for production deployment, and predicting Homer or Bart.

Show how to solve the cat and dog homework by preprocessing images in Google Colab, training a neural network, evaluating results, and introducing convolutional neural networks for better accuracy.

Learn the plan to apply convolutional neural networks to image classification using the Homer and Bart dataset, covering CNN theory, pre-processing, and implementation with Python and TensorFlow.

Explore the theory of convolutional neural networks and how they learn image features for object detection and facial recognition in computer vision, using convolution and pooling for efficiency.

Explore the convolution operation in a convolutional neural network by applying a 3x3 kernel to a 7x7 image to produce a feature map, then apply Relu activation and learn filters.

Pooling is the second step of a convolutional neural network, using multiple feature detectors to generate feature maps and apply max pooling to highlight the dog across varied images.

Learn how flattening turns max-pooled feature maps into a vector to feed a dense neural network, enabling a convolutional neural network to classify numbers with Relu and a sigmoid output.

Explore how a dense neural network, last step of convolutional networks, uses weights to activate output neurons and classify digits 1, 3, and 9 after flattening with relu.

Begin a Homer and Bart recognition project by importing libraries (matplotlib, seaborn, numpy, cv2) and building a convolutional neural network with sequential layers, pooling, flattening, and an image data generator.

Learn to mount google drive, unzip image datasets, and structure training and test folders for a convolutional neural network in TensorFlow, using tf.keras.preprocessing.image.load_image to preview Bart and Homer images.

Build train and test datasets with TensorFlow's image data generator, applying rescale normalization and preprocessing like rotation, horizontal flip, and zoom.

Build and train a convolutional neural network using sequential models, conv2d layers with 32 filters, relu activation, max pooling, flattening, dense layers, and categorical cross entropy for image classification.

Evaluate a neural network on the test data set using softmax outputs, arg max, and accuracy metrics, then analyze confusion matrices and classification reports to compare CNNs and dense nets.

Save the neural network structure in json and its weights with TensorFlow, then load the saved model to classify images in a computer vision workflow.

Learn to classify a single image by loading, pre-processing (resize to 64×64 and normalize to 0–1), and feeding it to the neural network to predict Bart or Homer.

Train a two-convolutional-layer cnn to classify cats and dogs using a 64×64 dataset, train for 10 epochs, and achieve about 72% accuracy on the test set.

Explore transfer learning and fine tuning for image classification with a Homer and Bart dataset, building advanced convolutional neural networks, extracting features, and comparing results across methods.

Learn transfer learning by reusing pretrained weights from an ImageNet base model to build a classifier, such as cats and dogs, while freezing general feature layers and training dense layers.

Apply transfer learning and fine-tuning for image classification on the Homer and Bart dataset using Google Colab, combining a base network with dense layers, pooling, and dropout.

this lecture extends transfer learning and fine tuning by defining train and test sets, applying rescaling, and preparing a 256 by 256 dataset with batch size eight for two classes.

Load a pre-trained neural network such as Resnet 50 from tf.keras applications, trained on image net, for transfer learning; freeze base layers and add a custom dense layer.

Implement a custom dense head for a frozen ResNet base, using flattening or global average pooling, add dropout, and train a two-class softmax output.

Connects the base model to dense head; uses ResNet architecture with global average pooling and dropout; train with adam and categorical cross entropy for 50 epochs, evaluate on test data.

Evaluate a neural network with transfer learning and compare results to prior approaches, achieving 0.81 accuracy on test data, and analyze bart versus homer with confusion matrix and precision-recall.

Learn the theory of fine tuning in transfer learning and how to transfer weights, train layers with a dense network, and use a small learning rate for differing images.

implement fine tuning by unfreezing the Bayes base model, selecting the last convolutional layers to train. compare results with transfer learning using 50 epochs and accuracy evaluation.

Save the neural network structure and weights, load the ResNet model, and classify a single image by resizing, normalizing, and predicting its class.

Describe a step-by-step homework solution for cat and dog classification using transfer learning with MobileNetV2 in Colab, including data preparation, model setup, training, evaluation, and fine-tuning.

Learn the plan to use convolutional neural networks to detect six basic emotions in images and videos, enabling richer human-machine interaction and practical applications.

Begin implementing emotion classification from images in Google Colab by importing OpenCV, NumPy, Matplotlib, Seaborn, and TensorFlow, and loading train and validation sets from Google Drive.

Create the train and test datasets for seven emotion classes using an image data generator with rescaling, augmentation, and 48 by 48 images, batch size 16, and categorical labels.

Build a convolutional neural network with stacked convolutional layers, batch normalization, pooling, dropout, and dense layers to classify seven emotion classes, then load weights for testing.

Save the neural network structure and weights, load the structure from json, and load weights from Google Drive to verify the module is correctly loaded.

Evaluate a convolutional neural network on a 3000-image test set, achieving 57% accuracy and analyzing recall and precision for seven emotion classes, including happy with 88% recall and 76% precision.

classify a single image by detecting a face with a haarcascade, extracting the face region, resizing to 48 by 48, normalizing, and predicting emotion with a neural network.

Classify emotions for multiple faces by detecting faces with a haarcascade detector, resizing each face, predicting emotions with a neural network, and annotating the image with bounding boxes and labels.

Classify emotions in videos by detecting faces in frames with OpenCV, predicting emotions, drawing bounding boxes, and saving the result to Google Drive.

Explore solving a two-class emotion detection task (angry vs happy) in a Google Colab workflow, training and validating a neural network, and evaluating accuracy, recall, and precision.

Outline the plan of attack on autoencoders, covering their theory and relation to neural networks, linear autoencoders for image compression on handwritten digits, and convolutional autoencoders for clothing images.

Learn how autoencoders encode inputs into a compressed representation and decode to reconstruct the original image, enabling denoising, image compression, and fraud detection via reconstruction error.

Explore autoencoders for image compression by implementing linear and convolutional models in TensorFlow, using the MNIST dataset in a Google Colab notebook and visualizing results.

Visualize MNIST images by selecting random samples from x_train, display them in a grayscale 10x10 grid, label each with its index, and prepare for the upcoming pre-processing phase.

Normalize image pixels from 0–255 to 0–1 to speed training and improve results, flattening 28×28 images to 784 vectors for 60k training and 10k testing samples in a linear autoencoder.

Build and train a linear autoencoder that compresses 784-pixel images to 32 features through a 128-64-32 encoder and decodes back to 784 pixels.

Build an encoder from the autoencoder to map 784-pixel images to a 32-dimensional code, then encode and reshape for visualization of the compressed representation.

Decode images using a linear autoencoder by constructing a custom decoder and linking 32-pixel encoded data through 64 and 128-pixel layers to the 28 by 28 grayscale image.

Explore encoding and decoding of test images using an encoder and decoder, compare original and decoded images, visualize results to evaluate autoencoder performance, preparing for convolutional autoencoders.

Learn to implement a convolutional autoencoder with conv2d, max pooling, upsampling, flatten, and reshape on fashion Mnist, including normalization of 28×28 grayscale images.

Build and train a convolutional autoencoder by defining an encoder and decoder with conv2d layers, pooling, strides, flattening, and upsampling to reconstruct 28x28 grayscale images.

Track how a convolutional autoencoder transforms a 28x28x1 image through 3x3 convolutions with stride 1, padding same or valid, and 2d pooling and upsampling, tracking shapes.

learn to extract 128-pixel encodings from 28x28 grayscale images using a convolutional autoencoder, encode test images, and reconstruct them with the full autoencoder, noting room for improvement on complex data.

Build and train a convolutional autoencoder on CIFAR-10 images in Google Colab, using 32x32x3 inputs and a 4x4x16 encoded representation, with visualization of original, coded, and decoded results.

Explore the plan of attack for YOLO, the fast object detector, covering its theory, Darknet image detection, and video applications for traffic lights, people, and cars.

Learn the basics of the YOLO architecture, including bounding box detection with no dense layers, frequent concatenation and residual blocks, and three-scale detection through upsampling, pooling, and three output layers.

Explore object detection with the darknet framework and yolov4 in google colab by cloning the darknet repo, compiling the library, and downloading pretrained yolov4 weights, linking to transfer learning concepts.

Test the object detector using the darknet framework and YOLO v4, run detections on sample images, and visualize results with cv2 and matplotlib.

Switch to gpu to speed up darknet detection, verify gpu usage, and configure makefile and OpenCV for coco, image net, or image datasets with YOLOv4, adjusting thresholds and exit outputs.

Explore how thresholds and ext outputs affect detection quality in a yolov4 darkenet setup, comparing high versus low thresholds and showing bounding box positions and sizes.

Implement object detection in videos using a YOLOv4 based detector, processing video frames from Google Drive. The model detects cars, people, traffic lights, and handbags across frames.

Detect objects in video using yolo, identifying people, backpack, laptop, and book frame by frame. Show the results demonstrate high accuracy and discuss commercial applications of this powerful technique.