Deep Learning and Reinforcement Learning with Tensorflow

This video provides an overview of the entire course.

The aim of the video is to familiarize viewers with reinforcement learning and common terminologies used in the reinforcement learning world with the help of real-life analogies.

• Understand the typical reinforcement learning setup

• Understand terminologies using a couple of real world analogies

• Draw similarities with the analogies and concluding what exactly is reinforcement learning

The aim of the video is to make the viewer understand the broad steps involved in reinforcement learning and give a hint to what kind of tasks can be solved with reinforcement learning.

• See common reinforcement learning use cases and recent notable achievements in the field

• Learn how to frame any task as a reinforcement learning problem

• Understand the steps involved in the reinforcement process and the kinds of reinforcement learning algorithms

Understanding how to use environments provided by the Open AI gym framework to try and test different reinforcement learning algorithms that are going to be implemented throughout the course.

• Explain Open AI gym and setting it up

• Understand the gym’s environment interface and common methods

• Create a dumb agent that works for different gym environments

The aim of the video is to familiarize viewers with the Taxi-v2 environment.

• Understand state space of the Taxi-v2 environment

Let’s create a dumb agent that operates on the Taxi-v2 environment and see how well it performs.

• Load the environment

• Make the dumb agent

• Measure the performance

In this video, we’re going to learn about reinforcement Q-Learning and we’ll use this technique to create a smart agent.

• Learn about the Q-Learning equation

Now that we’ve covered the theoretical aspect of Q-Learning, let’s implement the code for the same.

• Load the environment

• Initialize the Q-table

• Update the Q-values

Let’s evaluate our Q-Learning agent and calculate the performance metrics.

• Evaluate our agent by “exploitation” of q-table

• Calculate the performance metrics

The aim of the video is to familiarize viewers with the Cartpole environment.

• Understand the state space of the Cartpole environment

• Verify the environment with code

Let’s apply a new technique of Q-networks to balance the Cartpole in the Cartpole environment.

• Learn about the neural network architecture

In this video, we’ll go through some basic TensorFlow examples to understand how TensorFlow library works.

• See how TensorFlow works

• Install TensorFlow

• See how to calculate the product of two numbers in TensorFlow

Now we’ll implement Q-networks using TensorFlow.

• Take a look at the Q-network architecture

• See the Q-networks for Cartpole

Let’s restore the configuration parameters saved earlier and we’ll use them to see our Cartpole agent in action.

• Test the Q-network

• Compare the performance

Let’s start this section by understanding the DQN architecture.

• Take a look at target freezing

The aim of the video is to learn the steps involved in training a DQN.

• See components involved in DQN training

• Understand the algorithm

Now that we have seen the DQN training algorithm, let’s try and implement Deep Q-networks.

• Add layers

• Build the network

Let’s implement experience replay and the training logic, in this video.

• Take a look at training logic configurations

• See epsilon decay example run

• Calculate the loss function

In this video, we will implement couple of enhancements to our DQN approach.

• See the problem of over-estimation of Q-values

• Implement changes in the Double DQN code

The aim of the video is to familiarize the viewers with logging, saving and visualizing which are the ways to purses data and represent data in one way or the other.

• Understand the use of logging and implementing its metrics after every epoch

• Save model weights and parameters every once in a while

• Visualize results over the time by plotting graphs

The aim of the video to structure our code base nicely and go through our refactored version of previous written dueling double DQN code.

• Modularize code by dividing it into chunks of functional entities

• Focus upon each of the chunks independently

• Understand the directory structure of a modularized codebase

The aim of the video is to make viewer go through different ways to debug TensorFlow models and TensorFlow best practices.

• Debug TensorFlow with print statements

• Use API methods during debugging

• Experiment with different TensorFlow practices to avoid bugs

The aim of the video is to enable the viewers to leverage TensorFlow on distributed setups and non-CPU devices such as GPUs and TPUs.

• Learn about various devices compatible with TensorFlow such as CPU, GPU and TPU

• Distribute computation across multiple devices

• Understand hyperparameter optimization and parallelization

The aim of this final video is to show the ways to use the knowledge that we have gained so far in this course to do some exciting stuffs in reinforcement learning.

• Understand higher-level TensorFlow APIs such as Keras, TF-slim, Skflow, and Tflearn

• Move beyond existing Gym environments and trying other practical applications of reinforcement learning

• Explore other state-of-the-art implementations apart from DQNs