Reinforcement Learning with Pytorch
4.4 (281 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
2,003 students enrolled

Reinforcement Learning with Pytorch

Learn to apply Reinforcement Learning and Artificial Intelligence algorithms using Python, Pytorch and OpenAI Gym
4.4 (281 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
2,003 students enrolled
Created by Atamai AI Team
Last updated 8/2020
English
English [Auto]
Current price: $104.99 Original price: $149.99 Discount: 30% off
5 hours left at this price!
30-Day Money-Back Guarantee
This course includes
  • 7 hours on-demand video
  • 5 articles
  • 3 downloadable resources
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • Reinforcement Learning basics
  • Tabular methods
  • Bellman equation
  • Q Learning
  • Deep Reinforcement Learning
  • Learning from video input
Course content
Expand all 69 lectures 07:14:19
+ Welcome to the course
3 lectures 04:15
Before you start - Videos quality!
00:45
Resources
01:07
+ Introduction
9 lectures 44:07
Introduction #2
05:09
Introduction #3
04:33
Introduction #4
06:00
Lab. OpenAI Gym #1
03:06

Here to clarify the doubts and to have clear understanding about terms:

  • episode -  episode is single instance of a game - so in case of CartPole it is since when single game starts up to the moment when this game finishes... so according to explanation from OpenAI website it is :
    "The episode ends when the pole is more than 15 degrees from vertical, or the cart moves more than 2.4 units from the center."  

      So technically until done flag is set to True. 

      For example we do our training for 500 episodes... so we play game 500 times...


  • step - it is single move of our agent - so in case of CartPole it is single move left or right... for single episode (single instance of a game) - we have multiple steps (multiple agent's actions) to reach the main goal - not to let pole fall over.
Lab. OpenAI Gym #2
10:40
Lab. OpenAI Gym #3
02:57
Lab. OpenAI Gym #4
06:00
+ Tabular methods
19 lectures 02:08:27
Deterministic & Stochastic environments
07:28
Rewards
04:31
Bellman equation #1
06:11
Bellman equation #2
02:57
Resource - code
00:21
Lab. Algorithm for deterministic environments #1
10:23
Lab. Algorithm for deterministic environments #2
10:45
Lab. Algorithm for deterministic environments #3
10:34
Lab. Algorithm for deterministic environments #4
06:24
Lab. Test with stochastic environment
04:04
Q-Learning
10:08
Lab. Algorithm for stochastic environments
06:50
Exploration vs Exploitation
02:56
Lab. Egreedy
07:34
Lab. Adaptive egreedy
05:39
Bonus Lab. Value iteration
11:32
Homework
04:30
Homework. Solution
05:17
Homework. Tuning
10:23
+ Scaling up
12 lectures 01:21:00
Scaling up
06:04
Neural Networks review
04:00

Here it's worth mentioning that when we set random seed for our environment - in configuration provided in this video 

env.action_space.sample()

will still get fully random (not repeatable) results.

This is due to that fact that for sample moves for some reasons Gym uses different approach to get randomness. More details here:

https://github.com/openai/gym/blob/339415aa03a9b039a51f67798a44f8cd21464091/gym/spaces/box.py#L28-L29

So if you also need to "fix" randomness of sample moves - you have to use: 

from gym.spaces.prng import seed

seed(seed_value)

Lab. Neural Networks review #1
09:51
Lab. Neural Networks review #2
10:08
Lab. Random CartPole
06:45
Lab. Epsilon egreedy revisited
02:37
Lab. Pytorch updated ( version 0.4.0 )
07:35
Article. Pytorch updated! (further versions)
00:10
Lab. OpenAI Gym + Neural Network #1
10:04
Lab. OpenAI Gym + Neural Network #2
08:53
Lab. OpenAI Gym + Neural Network #3
04:35
Lab. Extended logging
10:18
+ DQN
9 lectures 58:49
Deep Reinforcement Learning
03:55
Lab. Deep Reinforcement Learning
05:48
Lab. Tuning challenge
01:39
Lab. Experience Replay #1
10:05
Lab. Experience Replay #2
09:25
Lab. Experience Replay #3
05:06
DQN
07:26
Lab. DQN
09:11
+ DQN Improvements
5 lectures 35:15
Double DQN
07:31
Lab. Double DQN
07:30
Dueling DQN
06:33
Lab. Dueling DQN
07:46
Lab. Dueling DQN Challenge
05:55
+ DQN with video output
11 lectures 01:20:00
CNN Review
05:53

File atari_wrappers.py (from OpenAI github page) has to be downloaded to the same location as all other files. It has to be in same directory - because then we will be able to import it directly in our code.

Lab. Random Pong
08:32
Saving & Loading the Model
01:19
Lab. Pong from video output #1
09:53
Lab. Pong from video output #2
09:01
Lab. Pong from video output #3
10:10
Lab. Pong from video output #4
10:13
Lab. Pong from video output #5
09:05

Here I also recommend giving a try with update_target_frequency = 2000. I noticed that sometimes it give even better results (game is resolved faster)!

Lab. Pong from video output #6
10:34
Potential improvements
04:09
Article. Stacking 4 images together
01:11
Requirements
  • Basic python knowledge is needed. AI / Machine Learning / Pytorch basics - nice to have but not fully necessary. Only open source tools will be in use.
Description

UPDATE:

All the code and installation instructions have been updated and verified to work with Pytorch 1.6 !!


Artificial Intelligence is dynamically edging its way into our lives. It is already broadly available and we use it - sometimes even not knowing it  - on daily basis. Soon it will be our permanent, every day companion.

And where can we place Reinforcement Learning in AI world? Definitely this is one of the most promising and fastest growing technologies that can eventually lead us to General Artificial Intelligence! We can see multiple examples where AI can achieve amazing results - from reaching super human level while playing games to solving real life problems (robotics, healthcare, etc).

Without a doubt it's worth to know and understand it!

And that's why this course has been created.

We will go through multiple topics, focusing on most important and practical details. We will start from very basic information, gradually building our understanding, and finally reaching the point where we will make our agent learn in human-like way - only from video input!

What's important - of course we need to cover some theory - but we will mainly focus on practical part. Goal is to understand WHY and HOW.

In order to evaluate our algorithms we will use environments from - very popular - OpenAI Gym. We will start from basic text games, through more complex ones, up to challenging Atari games

What will be covered during the course ? 

- Introduction to Reinforcement Learning

- Markov Decision Process

- Deterministic and stochastic environments

- Bellman Equation

- Q Learning

- Exploration vs Exploitation

- Scaling up

- Neural Networks as function approximators

- Deep Reinforcement Learning

- DQN

- Improvements to DQN

- Learning from video input

- Reproducing some of most popular RL solutions

- Tuning parameters and  general recommendations

See you in the class!

Who this course is for:
  • Anyone interested in artificial intelligence, data science, machine learning, deep learning and reinforcement learning.