Modern Reinforcement Learning: Deep Q Learning in PyTorch
4.5 (249 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
1,155 students enrolled

Modern Reinforcement Learning: Deep Q Learning in PyTorch

How to Turn Deep Reinforcement Learning Research Papers Into Agents That Beat Classic Atari Games
4.5 (249 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
1,157 students enrolled
Created by Phil Tabor
Last updated 5/2020
English
English [Auto]
Current price: $119.99 Original price: $199.99 Discount: 40% off
1 day left at this price!
30-Day Money-Back Guarantee
This course includes
  • 5.5 hours on-demand video
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • How to read and implement deep reinforcement learning papers
  • How to code Deep Q learning agents
  • How to Code Double Deep Q Learning Agents
  • How to Code Dueling Deep Q and Dueling Double Deep Q Learning Agents
  • How to write modular and extensible deep reinforcement learning software
  • How to automate hyperparameter tuning with command line arguments
Course content
Expand all 41 lectures 05:41:59
+ Fundamentals of Reinforcement Learning
6 lectures 01:00:46
Agents, Environments, and Actions
10:04
Markov Decision Processes
11:30
Value Functions, Action Value Functions, and the Bellman Equation
08:10
Model Free vs. Model Based Learning
03:34
The Explore-Exploit Dilemma
05:27
Temporal Difference Learning
22:01
+ Deep Learning Crash Course
7 lectures 55:06
Dealing with Continuous State Spaces with Deep Neural Networks
18:53
Naive Deep Q Learning in Code: Step 1 - Coding the Deep Q Network
07:55
Naive Deep Q Learning in Code: Step 2 - Coding the Agent Class
10:10
Naive Deep Q Learning in Code: Step 3 - Coding the Main Loop and Learning
09:21
Dealing with Screen Images with Convolutional Neural Networks
03:52
+ Human Level Control Through Deep Reinforcement Learning: From Paper to Code
14 lectures 01:54:18
How to Read Deep Learning Papers
07:15
Analyzing the Paper
20:33
How to Modify the OpenAI Gym Atari Environments
14:29
How to Preprocess the OpenAI Gym Atari Screen Images
02:55
How to Stack the Preprocessed Atari Screen Images
03:26
How to Combine All the Changes
01:30
How to Add Reward Clipping, Fire First, and No Ops
04:49
How to Code the Agent's Memory
10:55
How to Code the Deep Q Network
11:44
Coding the Deep Q Agent: Step 1 - Coding the Constructor
07:48
Coding the Deep Q Agent: Step 2 - Epsilon-Greedy Action Selection
02:22
Coding the Deep Q Agent: Step 3 - Memory, Model Saving and Network Copying
04:24
Coding the Deep Q Agent: Step 4 - The Agent's Learn Function
07:54
Coding the Deep Q Agent: Step 5 - The Main Loop and Analyzing the Performance
14:14
+ Deep Reinforcement Learning with Double Q Learning
2 lectures 24:30
Analyzing the Paper
15:39
Coding the Double Q Learning Agent and Analyzing Performance
08:51
+ Dueling Network Architectures for Deep Reinforcement Learning
4 lectures 33:08
Analyzing the Paper
14:01
Coding the Dueling Deep Q Network
03:21
Coding the Dueling Deep Q Learning Agent and Analyzing Performance
10:10
Coding the Dueling Double Deep Q Learning Agent and Analyzing Performance
05:36
+ Improving On Our Solutions
3 lectures 35:41
Implementing a Command Line Interface for Rapid Model Testing
09:30
Consolidating Our Code Base for Maximum Extensability
18:32
How to Test Our Agent and Watch it Play the Game in Real Time
07:39
+ Bonus Lecture
1 lecture 01:12
Bonus Video: Where to Go From Here
01:12
Requirements
  • Some College Calculus
  • Exposure To Deep Learning
  • Comfortable with Python
Description

In this complete deep reinforcement learning course you will learn a repeatable framework for reading and implementing deep reinforcement learning research papers. You will read the original papers that introduced the Deep Q learning, Double Deep Q learning, and Dueling Deep Q learning algorithms. You will then learn how to implement these in pythonic and concise PyTorch code, that can be extended to include any future deep Q learning algorithms. These algorithms will be used to solve a variety of environments from the Open AI gym's Atari library, including Pong, Breakout, and Bankheist.


You will learn the key to making these Deep Q Learning algorithms work, which is how to modify the Open AI Gym's Atari library to meet the specifications of the original Deep Q Learning papers. You will learn how to:

  • Repeat actions to reduce computational overhead

  • Rescale the Atari screen images to increase efficiency

  • Stack frames to give the Deep Q agent a sense of motion

  • Evaluate the Deep Q agent's performance with random no-ops to deal with model over training

  • Clip rewards to enable the Deep Q learning agent to generalize across Atari games with different score scales


If you do not have prior experience in reinforcement or deep reinforcement learning, that's no problem. Included in the course is a complete and concise course on the fundamentals of reinforcement learning. The introductory course in reinforcement learning will be taught in the context of solving the Frozen Lake environment from the Open AI Gym.

We will cover:

  • Markov decision processes

  • Temporal difference learning

  • The original Q learning algorithm

  • How to solve the Bellman equation

  • Value functions and action value functions

  • Model free vs. model based reinforcement learning

  • Solutions to the explore-exploit dilemma, including optimistic initial values and epsilon-greedy action selection

Also included is a mini course in deep learning using the PyTorch framework. This is geared for students who are familiar with the basic concepts of deep learning, but not the specifics, or those who are comfortable with deep learning in another framework, such as Tensorflow or Keras. You will learn how to code a deep neural network in Pytorch as well as how convolutional neural networks function. This will be put to use in implementing a naive Deep Q learning agent to solve the Cartpole problem from the Open AI gym. 

Who this course is for:
  • Python developers eager to learn about cutting edge deep reinforcement learning