Hands on Robotic Vision: OpenCV, PyTorch, YOLO, Simulation

Name: Hands on Robotic Vision: OpenCV, PyTorch, YOLO, Simulation
Rating: 4.7 (5 reviews)

Master OpenCV PyTorch, YOLO & Visual Servoing to build vision-guided robot arms, simulation or real hardware with python

Created bySaroj Debnath

Last updated 9/2026

English

English [Auto],

What you'll learn

Master OpenCV for real-world image processing: color spaces, edge detection, contours, blob detection, and camera calibration.
Build and train deep learning models with PyTorch — CNNs, YOLO, ResNet, and SAM — for object detection and segmentation.
Integrate computer vision with robot arms using eye-to-hand/eye-in-hand setups, visual servoing, and ArUco markers.
Deploy complete vision pipelines with FastAPI and simulate robot vision projects in PyBullet with Franka Panda.
Simulate Robots using Pybullets
Hands on Robotic Vision with Real Industrial Robot

Course content

13 sections • 110 lectures • 31h 32m total length

Introduction4:21
Setup and Resources27:34

Load Images34:39
Understand the color spaces15:45
Image Thresholding21:51
Canny Edge Detection15:24
Sobel Edge Detection8:35
Contour Detection10:12
Morphological Operation15:20
Template Matching7:10
Hough Transform12:17
Geometric Transformation7:57
Explore geometric transformation of images with resize, rotate (45° around the image center), translate, and flip using warp affine and rotation matrices.
Histogram Equalization9:16
Feature Matching12:22
Background Subtraction6:14
Camera Calibration8:57
Understand Homography and perspective transform10:31
Object Tracking16:34
Aruco Marker15:45
Detect Aruco markers with OpenCV in real time using a 6x6, 250 dictionary to read IDs and track 3D pose, enabling robot localization, augmented reality, and drone landing.
Optical Flow14:18
PROJECT1 : Augmented Reality23:10
PROJECT2 : Create Panoramic Scenes22:19

Introduction to Deep Learning9:49
Setting up the environment for deep learning22:09
Set up a full deep learning environment using Google Colab and local GPU, verify CUDA availability, benchmark CPU versus GPU, and inspect a simple neural network layer with torch info.
Understanding Neural Network7:32
How does a Neural Network learns?10:07
Learn how neural networks learn by converting random weights into useful predictions via the loss function and gradient descent. Understand forward and backward passes, backpropagation, and the weight update rule.
Let's understand Tensors25:06
Training a NN from scratch41:56
Learn to build and train a neural network from scratch in PyTorch, with a training loop, a custom moons data set, an MLP, BCE loss with logits, and Adam optimization.
PROJECT: MLP with mnist dataset27:20
Introduction to Convolutional Neural Network (CNN)14:25
How CNN works?15:03
CNNs Block by Block26:45
Explore how a color image passes through a three-block CNN with 3x3 convs, batch norm, relu, and max pooling, then global average pooling and a 10-class classifier on cipher 10.
Residual Connections16:06
Resnet31:49
Introduction to transfer learning11:45
Learn about EfficientnetB028:18
Data Augmentation42:27
PROJECT: Build a defect classifier59:43
Understand Object Detection27:07
Let's go more deep into Object Detection18:26
Data Annotation25:51
Understanding YOLO22:41
Training Custom Object Detection model39:01
Detection Vs Segmentation10:19
Train a custom object segmentation23:11
TRAIN YOUR CUSTOM MODEL---> NO CODE13:47

Pre-Section Talk3:23
Pybullet Basics15:53
Aruco Markers53:51
Robotic Arm Basic with Pybullet10:14
Camera Sensor8:39
Force Sensor15:36
Collision Check for the arm17:40
Forwards Kinematics Simulation12:55
Homogenous Matrices14:59
Compute a two-arm robot end-effector position using homogeneous matrices, combining rotations about z with translations along links, then verify with a PyBullet simulation.
Inverse Kinematics19:15
Learn about Jacobian Matrices13:32
Apply the jacobian matrix to relate end-effector position changes to joint angles in a two-DOF arm, using delta p and delta q through forward kinematics and pseudo inverse.
Load ABB1207:54
Load ABB_AX3:08
Load YUMI5:47
Load UR2:23
Load Epson2:44
PROJECT: Husky and Kuka robot Pick and Place21:57

Requirements

Basic Python programming (variables, loops, functions)
No prior robotics or computer vision experience needed — everything is taught from scratch
A computer capable of running Python 3 and Anaconda
Familiarity with running scripts from the command line
Windows is preferred

Description

Computer vision is one of the most in-demand skills in modern robotics — and this course gives you everything you need to master it, from the very first pixel to a fully vision-guided robot.

You will start with the fundamentals of image processing using OpenCV, learning how to work with color spaces, detect edges and contours, recognize ArUco markers, and calibrate cameras. These are the building blocks every robotics engineer needs.

From there, you will move into deep learning for vision. You will build convolutional neural networks from scratch in PyTorch, apply transfer learning with ResNet, detect objects in real time with YOLO, and perform instance segmentation. Every concept is tied to a practical robotics use case.

The course then bridges theory and hardware. You will learn how cameras are modeled mathematically, how to set up eye-to-hand and eye-in-hand configurations, and how to implement Image-Based Visual Servoing — the technique that lets a robot move based on what it sees.

Finally, you will build complete projects in PyBullet simulation using a Franka Panda robot arm, including color-based grasping, vision-guided pick and place, ArUco-based navigation. The course closes with real hardware projects on a Universal Robot arm.

By the end, you will have a portfolio of robot vision projects and the confidence to apply computer vision to real-world robotics challenges.

Requirements: Basic Python. No prior robotics or CV experience needed.

Who this course is for:

Robotics engineers and students who want to add computer vision skills to their toolkit
Python developers curious about applying machine learning to real-world visual tasks
Hobbyists and makers who want to build vision-guided robot projects at home
Professionals in manufacturing or automation looking to implement visual inspection systems
Anyone who has completed a basic Python course and wants a practical, project-driven path into robot vision

Hands on Robotic Vision: OpenCV, PyTorch, YOLO, Simulation

What you'll learn

Explore related topics

Course content

Introduction and Setup2 lectures • 32min

Basic Python11 lectures • 1hr 46min

Dive into Image Processing20 lectures • 4hr 49min

FastAPI Basics2 lectures • 30min

More Image processing project4 lectures • 1hr 47min

Deep Learning24 lectures • 9hr 31min

3D depth Sensing3 lectures • 1hr 33min

Understanding Basics of Robotic Arms3 lectures • 30min

Robotic Arms Simulation with Pybullet17 lectures • 3hr 50min

Robotic Vision Simulation5 lectures • 1hr 10min

Requirements

Description

Who this course is for: