
Explore practical reinforcement learning by building a simple robot with Lego Mindstorms, sensors, and an objective function, then program the agent to learn tasks within its environment.
Reinforcement learning mimics how humans learn by letting an agent interact with its environment to gain from rewards and penalties, unlike supervised or unsupervised learning.
Discover the reinforcement learning workflow with a self-driving car example. The agent uses state, action, and rewards to iteratively improve decisions in simulated and real environments.
Learn how the simplest reinforcement learning method, the multi-armed bandits algorithm, optimizes actions in uncertain robotics environments by using rewards and exploration to identify the best option over time.
Learn how to balance exploration and exploitation in reinforcement learning using an epsilon-greedy approach, exploring doors randomly while exploiting the best-known action to maximize rewards.
Frame robot behavior as a multi-armed bandit with epsilon-greedy and softmax policies, and design reward functions using infrared distance to obstacles to guide exploration and exploitation.
Implement a multi-armed bandit reinforcement learning on a Lego Mindstorms EV3 robot using Python, epsilon-greedy exploration, and three actions to learn obstacle distance via infrared.
Watch the robot learn autonomously through trial and error using a reward function. It demonstrates obstacle avoidance and action selection via a multi-armed bandit with epsilon-greedy exploration.
Install micro python on EV3 Mindstorms by flashing a microSD card with the micro python image built on ev3 dev debian linux, enabling the embedded environment.
Install Visual Studio Code and add the LEGO Mindstorms EV3 Python extension to write code and upload it to the brick, enabling control of sensors.
Set up Visual Studio Code with the EV3 micro Python image, create a Helloworld project, connect via USB, and download and run the code to beep.
Learn to read a touch sensor using Python 3, instantiate the sensor, and control LEDs to indicate when the sensor is pressed, with a looping program and timing considerations.
Learn to design an event-driven robot program using wait for press, wait for release, and wait for bump with timeouts; touch sensor events trigger subsequent code.
Control led lights by cycling colors and setting duration per color, including amber on the left for two seconds. Animate red and green police lights and adjust on/off intervals.
Sweep a target angle at a specified speed using the four degrees function, exemplified by 180 degrees at 50 percent speed, including blocking and non-blocking modes and direction reversal.
Learn to command a motor to complete a number of rotations or degrees at a speed, with 360 degrees equaling one full rotation and reverse direction by negating values.
Control motors for a specified time and speed using a function, and synchronize two motors with move tank to prevent timing differences in differential drive robots.
Diving into Reinforcement Learning can seem daunting if you don't have the proper hands-on guidance. Many times, people have asked me whether they should master Deep Learning before delving into Reinforcement Learning and my answer has always been that "it depends on what you want to do with RL". RL is a broad domain in its own respect. There are classical RL algorithms that can be learned and applied without any Deep Learning experience. There is also Deep Reinforcement Learning which leverages neural networks to help RL agents to learn proper behaviors in their environment through trials-and-error with reward functions.
This course has been designed to be the easiest and fastest basic entry point into RL and its applications in Robotics. From the first to the last videos, I explain every concept in RL in the context of robotics. I am intentional about this because I want to empower you to readily know when and how to apply RL techniques in Robotics. It is not an advanced course. Instead, it is your best option when you are getting started in RL (without any prior knowledge) and you are interested in being able to readily apply what you learn in your robotic projects.
Even though I use sensors and actuators from the EV3 Mindstorms robotics kit in the hands-on implementation sessions, you don't necessarily have to purpose the kit to get the best out of this course. It will certainly enhance your learning experience if you have the kit but don't worry if you don't. Without the kit, you can still understand the concepts and apply them on any robotic platform. It is my desire that after finishing this course, your passion for RL will be ignited and you will study further to know about more advanced algorithms and techniques.
So, while this course won't teach you how to build super-human capabilities into your robotics projects, you will certainly learn how to program robots to behave well in their environments without explicit instructions on what's considered "proper behavior".