ROS2 Self Driving Car with Deep Learning and Computer Vision

Create sign board meshes in Blender, export as obj, and import into Gazebo as models with sdf and config files, understanding Gazebo’s physics and gravity compared to Blender.

Create gazebo models from 3d meshes by joining a sign board to a stand with a fixed joint, fix gravity, and prepare a track for simulation with traffic lights.

Configure traffic lights for a Gazebo ROS2 self-driving car simulation by modeling red, yellow, and green lamps with cylinders and a light plug-in, with independent timers via a bash script.

Create dynamic traffic light models in Gazebo with a light plugin and an SDF model. Blink green, yellow, and red lights with defined intervals and ambient, diffuse, and specular lighting.

Publish velocity commands via a twist message on the command velocity topic to control the car's linear x and angular z in a 2d environment.

Create a Python ROS2 publisher to publish a command velocity twist message on the cmd_vel topic, drive a Prius in simulation, and configure the node with setup.py and console scripts.

Explore how self-driving cars operate without human intervention across five autonomy levels, and the four-step process: mapping, scanning, anticipating, and responding, using sensors to safely reach a destination.

Explore building a self-driving car in simulation using deep neural networks and sensor fusion to follow lanes, navigate intersections, comply with speed limits, and stop at signs.

Break down self-driving process into detection and control, detailing lane lines, sign, and traffic light detection using segmentation, estimation, and data extraction, plus a rules-based controller for steering and speed.

Develop lane-follow capability by detecting lane lines through color-based segmentation, filling gaps with estimation, and training before advancing to the control stage.

Explore estimation techniques to transform boxed patches from middle lane segmentation into an accurate boundary representation, avoiding false positives and refining lane detection in autonomous driving.

We extract information from the lead to reduce testing parameters and define our first rule-based control algorithm named after the feature, to keep the car following the lead, follow live.

Implement the lane assist control system by defining a follow-lead function that estimates the required turn angle from distance and curvature, then clamps to max angles and interpolates for steering.

Object detection works in two sequential steps: localization and classification. Localization marks signs in every frame, then classification identifies the sign, such as a speed sign.

Apply the Hough circles method to localize circular road signs by analyzing edge gradients in grayscale images, using adjustable radius, distance, and vote thresholds to detect circles.

Implement an optical flow based sign tracker that transitions from detection to tracking, initializes with good features in a region of interest, and uses Lucas parameters to track across frames.

Explore why traffic light detection cannot reuse sign detection algorithms, highlighting localization challenges and the need for multiple features and cnn-based separate models for red, green, and yellow.

Learn the theory of haar cascades for traffic light detection, focusing on features, integral images, and cascade classifiers to achieve fast, low-cost object detection on vehicle platforms.

Build a traffic light detection system with Haar cascades: extract frames, balance positive/negative data, annotate positives, and compare annotated versus augmented training results.

Train a traffic light detector with a haar cascade on windows, from gathering training videos to annotating positives, generating samples, and training with OpenCV, then test on video.

Build a traffic light detection pipeline with Haar cascades in Linux, loading the trained cascade, detecting lights in grayscale, and highlighting them with rectangles in a driving simulation.

implements a confirmation and state retrieval mechanism for traffic lights in ROS2 self-driving car project, using color segmentation and circular-region analysis to confirm detection and determine traffic light state.

Review the traffic light detection process flow: frames are analyzed, a green heart cascade detects lights, confirm and track with optical flow, extract region of interest, recycle on failure.

Discover how satnav systems locate a car using an overhead view, localize with multiple satellites, map road networks, compute the shortest route, and enable motion planning for a self-driving car.

Develop a four-stage satnav for a self-driving car, featuring background subtraction localization, occupancy grid creation, graph of interest points, Dijkstra path computation, and a motion planner yielding steering and speed.

Design a satellite navigation integrated self-driving car by implementing SDC v2 with a GPS navigator, upper camera processing, and dynamic engine control, plus enhanced board localization with road network refinement.

Explore stage 1 localization of the sat nav feature in a ros2 self-driving car, covering localization module tweaks, gps navigation integration, and runtime lane segmentation debugging in simulation.

Explore stage two mapping in sat-nav by transforming an occupancy grid into a graph of areas of interest and paths, using user-selected start and destination points.

Modify the SAT NAV stage two mapping to compute start and end points from user input and refine decision points to remove false positives, using closest node and centroid methods.

Explores stage four motion planning for ROS2 self-driving cars, removing obstacle navigation, adding a rolling average filter, and implementing a sat nav enable toggle for improved control.