Udemy
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Development
Web Development Data Science Mobile Development Programming Languages Game Development Database Design & Development Software Testing Software Engineering Software Development Tools No-Code Development
Business
Entrepreneurship Communication Management Sales Business Strategy Operations Project Management Business Law Business Analytics & Intelligence Human Resources Industry E-Commerce Media Real Estate Other Business
Finance & Accounting
Accounting & Bookkeeping Compliance Cryptocurrency & Blockchain Economics Finance Finance Cert & Exam Prep Financial Modeling & Analysis Investing & Trading Money Management Tools Taxes Other Finance & Accounting
IT & Software
IT Certifications Network & Security Hardware Operating Systems & Servers Other IT & Software
Office Productivity
Microsoft Apple Google SAP Oracle Other Office Productivity
Personal Development
Personal Transformation Personal Productivity Leadership Career Development Parenting & Relationships Happiness Esoteric Practices Religion & Spirituality Personal Brand Building Creativity Influence Self Esteem & Confidence Stress Management Memory & Study Skills Motivation Other Personal Development
Design
Web Design Graphic Design & Illustration Design Tools User Experience Design Game Design 3D & Animation Fashion Design Architectural Design Interior Design Other Design
Marketing
Digital Marketing Search Engine Optimization Social Media Marketing Branding Marketing Fundamentals Marketing Analytics & Automation Public Relations Paid Advertising Video & Mobile Marketing Content Marketing Growth Hacking Affiliate Marketing Product Marketing Other Marketing
Lifestyle
Arts & Crafts Beauty & Makeup Esoteric Practices Food & Beverage Gaming Home Improvement & Gardening Pet Care & Training Travel Other Lifestyle
Photography & Video
Digital Photography Photography Portrait Photography Photography Tools Commercial Photography Video Design Other Photography & Video
Health & Fitness
Fitness General Health Sports Nutrition & Diet Yoga Mental Health Martial Arts & Self Defense Safety & First Aid Dance Meditation Other Health & Fitness
Music
Instruments Music Production Music Fundamentals Vocal Music Techniques Music Software Other Music
Teaching & Academics
Engineering Humanities Math Science Online Education Social Science Language Learning Teacher Training Test Prep Other Teaching & Academics
Web Development JavaScript React Angular CSS Node.Js PHP HTML5 Vue JS
AWS Certification Microsoft Certification AWS Certified Solutions Architect - Associate AWS Certified Cloud Practitioner CompTIA A+ Amazon AWS Cisco CCNA CompTIA Security+ Microsoft AZ-900
Microsoft Power BI SQL Tableau Data Modeling Business Analysis Business Intelligence MySQL Qlik Sense Data Analysis
Unity Unreal Engine Game Development Fundamentals C# 3D Game Development C++ Unreal Engine Blueprints 2D Game Development Mobile Game Development
Google Flutter iOS Development Android Development Swift React Native Dart (programming language) Kotlin Mobile App Development SwiftUI
Graphic Design Photoshop Adobe Illustrator Drawing Digital Painting Canva InDesign Character Design Procreate Digital Illustration App
Life Coach Training Personal Development Neuro-Linguistic Programming Personal Transformation Life Purpose Mindfulness Sound Therapy Coaching CBT Cognitive Behavioral Therapy
Business Fundamentals Entrepreneurship Fundamentals Freelancing Business Strategy Startup Business Plan Online Business Blogging Leadership
Digital Marketing Social Media Marketing Marketing Strategy Internet Marketing Google Analytics Copywriting Email Marketing Startup YouTube Marketing

DevelopmentData ScienceReinforcement Learning

Modern Reinforcement Learning: Actor-Critic Algorithms

How to Implement Cutting Edge Artificial Intelligence Research Papers in the Open AI Gym Using the PyTorch Framework
Rating: 4.5 out of 54.5 (339 ratings)
2,201 students
Created by Phil Tabor
Last updated 10/2020
English
English [Auto]

What you'll learn

  • How to code policy gradient methods in PyTorch
  • How to code Deep Deterministic Policy Gradients (DDPG) in PyTorch
  • How to code Twin Delayed Deep Deterministic Policy Gradients (TD3) in PyTorch
  • How to code actor critic algorithms in PyTorch
  • How to implement cutting edge artificial intelligence research papers in Python

Requirements

  • Understanding of college level calculus
  • Prior courses in reinforcement learning
  • Able to code deep neural networks independently

Description

In this advanced course on deep reinforcement learning, you will learn how to implement policy gradient, actor critic, deep deterministic policy gradient (DDPG), twin delayed deep deterministic policy gradient (TD3), and soft actor critic (SAC) algorithms in a variety of challenging environments from the Open AI gym. There will be a strong focus on dealing with environments with continuous action spaces, which is of particular interest for those looking to do research into robotic control with deep reinforcement learning.

Rather than being a course that spoon feeds the student, here you are going to learn to read deep reinforcement learning research papers on your own, and implement them from scratch. You will learn a repeatable framework for quickly implementing the algorithms in advanced research papers. Mastering the content in this course will be a quantum leap in your capabilities as an artificial intelligence engineer, and will put you in a league of your own among students who are reliant on others to break down complex ideas for them.

Fear not, if it's been a while since your last reinforcement learning course, we will begin with a briskly paced review of core topics.

The course begins with a practical review of the fundamentals of reinforcement learning, including topics such as:

  • The Bellman Equation

  • Markov Decision Processes

  • Monte Carlo Prediction

  • Monte Carlo Control

  • Temporal Difference Prediction TD(0)

  • Temporal Difference Control with Q Learning

And moves straight into coding up our first agent: a blackjack playing artificial intelligence. From there we will progress to teaching an agent to balance the cart pole using Q learning.

After mastering the fundamentals, the pace quickens, and we move straight into an introduction to policy gradient methods. We cover the REINFORCE algorithm, and use it to teach an artificial intelligence to land on the moon in the lunar lander environment from the Open AI gym. Next we progress to coding up the one step actor critic algorithm, to again beat the lunar lander.

With the fundamentals out of the way, we move on to our harder projects: implementing deep reinforcement learning research papers. We will start with Deep Deterministic Policy Gradients (DDPG), which is an algorithm for teaching robots to excel at a variety of continuous control tasks. DDPG combines many of the advances of Deep Q Learning with traditional actor critic methods to achieve state of the art results in environments with continuous action spaces.

Next, we implement a state of the art artificial intelligence algorithm: Twin Delayed Deep Deterministic Policy Gradients (TD3). This algorithm sets a new benchmark for performance in continuous robotic control tasks, and we will demonstrate world class performance in the Bipedal Walker environment from the Open AI gym. TD3 is based on the DDPG algorithm, but addresses a number of approximation issues that result in poor performance in DDPG and other actor critic algorithms.

Finally, we will implement the soft actor critic algorithm (SAC). SAC approaches deep reinforcement learning from a totally different angle: by considering entropy maximization, rather than score maximization, as a viable objective. This results in increased exploration by our agent, and world class performance in a number of important Open AI Gym environments.

By the end of the course, you will know the answers to the following fundamental questions in Actor-Critic methods:

  • Why should we bother with actor critic methods when deep Q learning is so successful?

  • Can the advances in deep Q learning be used in other fields of reinforcement learning?

  • How can we solve the explore-exploit dilemma with a deterministic policy?

  • How do we get and deal with overestimation bias in actor-critic methods?

  • How do we deal with the inherent approximation errors in deep neural networks?

This course is for the highly motivated and advanced student. To succeed, you must have prior course work in all the following topics:

  • College level calculus

  • Reinforcement learning

  • Deep learning

The pace of the course is brisk and the topics are at the cutting edge of deep reinforcement learning research, but the payoff is that you will come out knowing how to read research papers and turn them into functional code as quickly as possible. You'll never have to rely on dodgy medium blog posts again.

Who this course is for:

  • Advanced students of artificial intelligence who want to implement state of the art academic research papers

Instructor

Phil Tabor
Machine Learning Engineer
Phil Tabor
  • 4.6 Instructor Rating
  • 1,170 Reviews
  • 5,117 Students
  • 4 Courses

In 2012 I received my PhD in experimental condensed matter physics from West Virginia University. Following that I was a dry etch process engineer for Intel Corporation, where I leveraged big data to make essential process improvements for mission critical products. After leaving Intel in 2015, I have worked as a contract and freelance deep learning and artificial intelligence engineer.

Top companies choose Udemy Business to build in-demand career skills.
NasdaqVolkswagenBoxNetAppEventbrite
  • Udemy Business
  • Teach on Udemy
  • Get the app
  • About us
  • Contact us
  • Careers
  • Blog
  • Help and Support
  • Affiliate
  • Investors
  • Impressum Kontakt
  • Terms
  • Privacy policy
  • Cookie settings
  • Sitemap
  • Accessibility statement
Udemy
© 2022 Udemy, Inc.