Automotive Camera [Computer vision, Deep learning] - 2A

Learn a Python-based camera perception pipeline for automotive vision, processing data to perform object detection and classification of road users, then track multiple objects with 2D bounding boxes and IDs.

Download and unzip the data folder from the resource section, and create a camera_perception directory. It contains two scenes, scene01 and scene02, with front camera PNGs at 10 fps.

Introduce the image preprocessing module for multi-camera setups, with camera position enum, timestamp, image data structure, and image reader, all building toward a complete UML diagram.

Define the camera position enumeration class with positions front left, front, front right, back, left, back right, left side middle, right side middle; include an invalid position at index 0.

Define a camera position enum class in Python with enum.Enum, enumerate valid and invalid poses, and test via a simple script while organizing imports in a package with __init__.py.

Implement a Python timestamp class with epoch seconds and milliseconds, use validated getters/setters, expose a date time computed property via UTC, and test with a dedicated test script.

Learn to implement a timestamp class with repr formatting that prints date-time and epoch values, and support subtraction to compute delta time in seconds between sequential images, with type checks.

Explore the image class concept as a data structure with frame id, timestamp, and rgb NumPy array. Define width, height, and channels, and implement init and repr methods.

Implement an image class as a data structure with frame id, timestamp, and numpy image data, compute width, height, and channels, and provide getters and setters with validation.

Define an image class as a data structure for camera frames, storing frame id, timestamp, and an nd array with height, width, and channels. Provide a visualization and debugging representation.

Implement image reader class initialized with a camera position, reading images from disk via a static method and returning an image structure with a timestamp in the image processing module.

Create an image reader in the image process package to read images from disk or camera, using a camera position enum, and return a numpy array via a static method.

Read image data into a numpy nd array, extract frame id and epoch time from the file name, and assemble an image data structure with epoch seconds and milliseconds.

Create a visualizer class to display image data and optionally overlay the frame id on a black rectangle with white text using cv2 imshow and waitKey.

Test the image preprocessing module by running a test script that uses the visualizer, reads images from the image reader, and prints frame IDs for debugging.

Learn to detect objects in camera images using 2d bounding boxes, defined by xmin, ymin, width, height, confidence, and category, with multiple boxes per image.

Define an object category enum and data structures in the object detector package, focusing on road user classes: person, bicycle, motorbike, car, bus, truck, plus an unknown placeholder for initialization.

Define image object 2d to hold a bounding box with x min, y min, width, height, object category and confidence; implement init, getters, setters, and repr with integer pixel validation.

Define an image object 2D list to hold multiple bounding boxes from an object detector, preserving frame id and timestamp with type checks and a total objects property.

Fix a setter typo that triggers a constant attribute error and demonstrate how the repr method prints epoch time, frame id, and total objects in a for loop.

Compare four-parameter representations of a 2d bounding box—xmin, ymin, width, height or x1, y1, x2, y2—and center-based formats, and discuss pixel versus normalized 0–1 values.

Explore designing an extensible object detector module with a factory pattern, an abstract detector interface, input pre-processing, visualization, and multi-format outputs for models like Faster RCNN, SSD, and YOLO.

Define an object detector abstract class with fixed road-user categories from COCO and two abstract methods: apply object detector on an image and get object list.

Implement the object detector abstract class as a template for detectors using Python's abc meta, with two abstract methods: apply_object_detector and get_objects, initializing valid categories from the object category enum.

Define the faster rcnn object detector class as a child of the abstract object detector, implementing initializer, apply object detector, and get object list with threshold, torch model, and device.

Learn to implement Faster R-CNN as the first object detector using pretrained PyTorch models from TorchVision, with CPU or GPU deployment, thresholding, and COCO labels for automotive camera perception.

Learn how to implement faster rcnn object detector: perform image pixel extraction and tensor conversion, run inference, apply post-processing, filter by categories and threshold, and output 2d objects with confidence.

Define an object detector type enumeration to catalog products, from faster RCNN with resnet 50 and SSD with the VGG16 backbone to Yolov 5 and Yolov 8 variants.

Create the object detector type enumeration to enumerate detector variants, starting with faster rcnn (resnet50) and outlining vgg16 with ssd and multiple yolov5 and yolov8 models.

Understand the object detector concept as a runtime factory selects a detector type. Expose init and detect objects methods that output bounding boxes with IDs and timestamps.

Implement the object detector class as a wrapper that selects the detector type from a factory, validates the threshold (default 0.7), and exposes a detect objects interface.

Test a complete faster rcnn pipeline by initializing the object detector with pretrained weights and processing a test image. Print detected objects with confidence scores and prepare later for visualization.

Extend the visualizer class to draw 2d bounding boxes on camera images via a static method, optionally showing class and score labels for object detection with faster rcnn.

Define a file writer and enum to store per-image faster RCNN outputs in JSON or YAML. This enables post-processing, ground-truth validation, and training data generation.

Define a file type enum class with json, xml, and csv, then implement a file writer in the output interfaces package.

Explore implementing a filewriter class that outputs per-image json files for detected objects, including bounding boxes, category, confidence, and timestamp, with automatic folder creation and json serialization.

Learn the SSD with VGG16 backbone as the next object detector in automotive camera course; compare it to Faster R-CNN and understand pretrained models, backbones, and apply object detector module.

Implement ssd 300 with vgg16 backbone for an object detector class, patch the detector framework, and compare its performance with faster rcnn in automotive scenes.

Explore how yolov5 defines nano to extra large model sizes and loads corresponding pretrained weights (.pt). Learn to configure the detector class with model size, thresholds, and nms settings.

Implement a Yolov 5 model size enumeration in a Python file YOLOV5.py by importing Enum and defining Nano, small, medium, large, and extra large as options.

Implement a Yolov5 class that loads a model size from ultralytics yolov5 via torch hub, sets confidence and NMS thresholds, and runs inference on images, producing detections with class labels.

Explore the YOLOv8 implementation, mirroring YOLOv5 with two classes for model size and detector. Identify the new model size enum and the file name extension difference, while keeping torch components.

Implement a YOLOv8 model size enumeration class in Python by copying the existing v5 code, adapting it from 5 to 8, and importing it into the detector initialization.

Implements the Yolov8 class for an object detector with Ultralytics, supporting nano, small, medium, large, and extra-large models, resizing to 640×640 for inference and bounding boxes with threshold and nms.

Define a top level camera perception class as the main interfacing API, enabling image pre processing and object detection with configurable parameters including folder path and detector type.

Define a top level camera perception API that orchestrates image pre processing, object detection, and visualization with configurable parameters, validations, and file output ready to use.

Implement the run method of the camera perception class, reading image paths, performing object detection, and conditionally printing, visualizing, or writing outputs based on console flags.

Test the complete camera perception pipeline via the main.py, configuring a front camera, Yolov8 small or Yolov5 medium detectors, JSON output, and optional console and raw image visualization.

Revisit the complete UML class diagram to understand the software architecture of the camera perception module, its independent modules, class dependencies, and the object detectors factory with Yolov5 and Yolov8 variants.

Validate object detectors against ground truth by comparing bounding boxes with IOU thresholds, identify true positives, false positives, false negatives, and report precision, recall, F1, aided by COCO API.