Vehicle Suspension Control 4: PID tuning with AI - TD3

Name: Vehicle Suspension Control 4: PID tuning with AI - TD3
Rating: 5.0 (6 reviews)

Master PID Tuning for Vehicle Suspension Control with State-of-the-Art TD3 Reinforcement Learning Algorithm in Python

Created byMark Misin Engineering

Last updated 2/2026

English

English [Auto],

What you'll learn

Get intuition in Backpropagation in Neural Networks
Apply Q-learning in Python in a simple grid-world.
Apply TD3 Reinforcement algorithm to suspension control
Get familiar with the PyTorch library

Course content

3 sections • 17 lectures • 2h 18m total length

Introduction0:59
Apply Td3 reinforcement learning to pid tuning for vehicle suspension, with PyTorch implementation; cover manual neural network training via gradient descent and Q-learning in gridworld.
Installation of Python & its libraries (Windows 11)5:25
Install Python and libraries on Windows 11, add Python to path, test with Hello World, then use pip to install numpy, matplotlib, and essential tools for Python-based control tutorials.
Installation of PyTorch (Windows 11)0:24
Revision of PID14:43
Explore how a proportional-integral-derivative controller uses its three terms to minimize error, overshoot, and steady-state error, with intuition from a rocket example and tuning insights.

Explanation of Backpropagation in Neural Networks8:07
Learn how backpropagation trains a neural network using forward passes to compute outputs, then backward gradient descent updates weights and biases to minimize the error toward target outputs.
Explanation of Q-learning in a grid world14:40
Explore how a Q-learning agent learns the optimal path in a 4x4 grid, using a 16-state, 4-action Q-table, random exploration, and iterative updates over 10,000 iterations.
Python codes: Backpropagation + Q-learning0:05

General overview of the code 115:13
Explore how TD3 applies to a vehicle suspension system using PyTorch, detailing the six networks, target updates, action noise, and PID constants as the agent’s action.
General overview of the code 212:51
Build a replay buffer with a deck array, fixed size after one million, and sample experiences to train TD3 actor and critic networks in PyTorch for PID tuning.
Main loop + some results14:59
Demonstrate td3-based PID tuning for a vehicle suspension by treating the plant as a black box, using PyTorch, replay buffers, and episode training to learn six PID gains.
Mathematics of a speed bump8:55
Model the speed bump as a half circle to describe wheel height over time, using a time-based piecewise y function and base excitation for a spring-damper suspension.
Environment class7:16
Explore the environment class that controls the suspension simulation, detailing reset, initial state, step function with pid gains, and reward computation using the half-car model.
Neural Networks classes10:30
Demonstrates the WeightsInIt initialization in the actor, outlines a 5-layer network for pid with 8 inputs and 6 outputs, and explains the forward pass with relu and tanh in td3.
Replay Buffer + TD3 class 18:23
Build and train TD3 agents using replay buffers, regular and target actors and critics, with batch sampling, proper PyTorch formatting, and GPU or CPU training.
TD3 class 2 - training method11:29
Train TD3 by sampling 100 experiences, computing target Q values from two critics, taking their minimum, adding noise to target actions, and updating critics and the actor via gradient descent.
Deployment4:51
Deploy a TD3-based adaptive PID tuning for vehicle suspension by loading the trained actor from a .pth checkpoint, extracting PID constants, and simulating the half-car model with state-space equations.
Python codes: TD3 training + deployment0:04

Requirements

Basics in Python

Description

Dive into PID tuning with AI using TD3 (Twin Delayed Deep Deterministic Policy Gradient) for vehicle suspension control! This hands-on course equips control systems engineers, robotics enthusiasts, and Python developers with practical skills to optimize half-car suspension models through reinforcement learning (RL).

Start with the fundamentals: implement simple Q-learning in a grid world environment using Python and NumPy. You'll code agents that learn optimal policies step-by-step, building intuition for RL basics like value functions.

Then, advance to real-world applications. Apply TD3, a state-of-the-art actor-critic algorithm, to tune PID controllers for a dynamic half-car model. Simulate vehicle suspension dynamics—handling bounce, pitch, and road disturbances. Line-by-line code explanations reveal how TD3 reduces overshoot and settling time in active suspension systems.

Perfect for automotive engineering and PyTorch RL practitioners. No prior RL experience needed—just basic Python and control theory.

What you'll get:

Complete, runnable Python code for Q-learning and TD3-PID tuning
Half-car model simulations with visualizations
Tips for deploying RL-tuned controllers in real-time systems

Boost your resume with AI-driven control systems expertise. Enroll now and transform theory into tunable, high-performance suspensions.

In addition, we will apply a simple backpropagation algorithm to manually tune a small neural network. It will give you necessary intuition in how neural nets are structured, how they are trained, and eventually, how they are deployed.

Who this course is for:

Engineering students
Engineering professionals

Vehicle Suspension Control 4: PID tuning with AI - TD3

What you'll learn

Explore related topics

Course content

Introduction: Python installation + PID revision4 lectures • 22min

Code explanation: Backpropagation + Q-learning3 lectures • 23min

PID tuning with the TD3 Reinforcement Learning algorithm (Coming Soon)10 lectures • 1hr 35min

Requirements

Description

Who this course is for: