Learning Path: R: Reward-Based Learning with R
0.0 (0 ratings)
Instead of using a simple lifetime average, Udemy calculates a course's star rating by considering a number of different factors such as the number of ratings, the age of ratings, and the likelihood of fraudulent ratings.
0 students enrolled
Wishlisted Wishlist

Please confirm that you want to add Learning Path: R: Reward-Based Learning with R to your Wishlist.

Add to Wishlist

Learning Path: R: Reward-Based Learning with R

Tackle programming problems and explore model-based and model-free learning algorithms for reward-based learning in R
0.0 (0 ratings)
Instead of using a simple lifetime average, Udemy calculates a course's star rating by considering a number of different factors such as the number of ratings, the age of ratings, and the likelihood of fraudulent ratings.
0 students enrolled
Created by Packt Publishing
Last updated 9/2017
English [Auto-generated]
Current price: $10 Original price: $200 Discount: 95% off
5 hours left at this price!
30-Day Money-Back Guarantee
  • 4 hours on-demand video
  • 1 Supplemental Resource
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
What Will I Learn?
  • Get to know the nuts and bolts of writing R code in RStudio
  • Get a tour of the most important data structures in R
  • Execute environment and Q-Learning functions with R
  • Learn episode and state-action functions in R
  • Master Q-Learning with Greedy Selection examples in R
  • Explore simulated annealing changed discount factor through examples in R
View Curriculum
  • Basic programming knowledge
  • Basic knowledge of math and statistics would be beneficial

R is a high-level statistical language and is widely used among statisticians and data miners to develop statistical applications. If you want to learn reward-based learning with R, then you should surely go for this Learning Path.

Packt’s Video Learning Paths are a series of individual video products put together in a logical and stepwise manner such that each video builds on the skills learned in the video before it.

The highlights of this Learning Path are:

  • Tour of the most important data structures in R
  • Implement applications for model-based and model-free RL

Beginning with the basics of R programming, this Learning Path provides step-by-step resources and time-saving methods to help you solve programming problems efficiently. You will be able to boost your productivity with the most popular R packages and data structures such as matrices, lists, and factors. You will be able to tackle issues with data input/output and will learn to work with strings and dates.

Moving ahead, you will know the differences in model-free and model-based approaches to reinforcement learning. This Learning Path discusses the characteristics, advantages and disadvantages, and typical examples of model-free and model-based approaches.You will learn Monte Carlo approach, Q-Learning approach, SARSA approach, and many more. Finally, you will take a look at model-free simulated annealing and more Q-Learning algorithms.

By the end of this Learning Path, you will be able to build actions, rewards, and punishments through these models in R for reinforcement learning.

About the Author

For this course, we have the best works of this esteemed authors:

  • Dr David Wilkins is a biologist with nearly a decade of experience writing R for research applications, particularly high-throughput analysis of genetic data. He has also developed a number of open source R packages.
  • Dr. Geoffrey Hubona held a full-time tenure-track, tenured, assistant and associate professor faculty positions at three major state universities in the Eastern United States from 1993-2010. In these positions, he taught dozens of various statistics, business information systems, and computer science courses to undergraduate, master's and Ph.D. students. Dr. Hubona earned a Ph.D. in Business Administration (Information Systems and Computer Science) from the University of South Florida (USF) in Tampa, FL (1993); an MA in Economics (1990), also from USF; an MBA in Finance (1979) from George Mason University in Fairfax, VA; and a BA in Psychology (1972) from the University of Virginia in Charlottesville, VA.
Who is the target audience?
  • This Learning Path is for programmers, data analyst, or data science enthusiasts who want to learn reward-based learning with R. No prior R knowledge is required as the Learning Path covers the fundamental concepts of R.
Students Who Viewed This Course Also Viewed
Curriculum For This Course
32 Lectures
Learn R programming
17 Lectures 01:29:14

This video provides an overview of the entire course.

Preview 02:09

In this video, we will see how to download and install RStudio, and set it up as an R editing environment.

Setting Up RStudio

In this video, we will Learn how to write, run, save and load R scripts in the RStudio source pane.

Writing, Running, and Saving R Scripts

In this video, we will understand how to use numbers and perform arithmetic operations in R.

Preview 04:36

The aim of this video is to make us understand how to create and use R variables, and the basics of vectors and vectorised operations.

Working with Variables and Vectors

In this video, we will understand how to find and use functions.

Using Functions and Reading Function Documentation

In this video, we will see what data types are and how to work with vectors of different data types.

Exploring Vectors in Depth and Understanding Data Types

This video explains us what is the purpose and properties of matrices and arrays and how to create them, and how to subset elements from them.

Working with Matrices and Arrays

The aim of this video is to make us understand what list data structure is and how does list differ from vectors. 

Discovering Lists

In this video, we will understand how to use data frame as a flexible way to represent and work with tabular data in R.

Discovering Data Frames

This video explains us why factors exist and how to use them in base R.

Exploring Factors

Datasets are often provided to you in a delimited format such as CSV (comma-separated value). In this video, we will learn how to load data from this and other delimited formats into R.

Reading Data from a File

When working with data, it’s often useful to subset a data frame by value. In this video, we will learn how to combine logical operators with data frame subsetting to subset datasets by value.

Subsetting Data Frames

Large data sets can be difficult to understand at a glance. This video aims to explain how to apply a range of statistical summary functions to condense key statistical properties from dataset variables.

Statistical Summaries of Data

Although there are hundreds of statistical tests that can be performed in R, many of them are applied according to a similar pattern. In this video, we will learn how to perform three common statistical tests in two different ways.

Statistical Tests on Data

Data sets will not always contain all the information you need. In this video, we will learn how to manipulate and combine variables to reshape a data set for your application.

Manipulating Data

When you finish working with a data frame, you need to write it back to file to work with it later or pass to somebody else. In this 

video, we will learn how to write a data frame to file.

Writing Data to File
Discover Algorithms for Reward-Based Learning in R
15 Lectures 02:36:00

This video provides an overview of the entire course.

Preview 05:51

How do you represent the environment when you have no explicit MDP model?

  • Determine the rules, “Physics,” structure of the state space
  • Determine the possible states, actions, new states, and rewards, and what you need to do once you have determined all of that
  • Build an environment function in R 
R Example – Building Model-Free Environment

How do you determine the optimal policy to “Solve” your reinforcement learning problem?

  • Observe State-Action-New-State reward experience data
  • Use this data to determine highest-value actions for each state 
R Example – Finding Model-Free Policy

In this video, we will continue with the optimal policy to “Solve” your reinforcement learning problem.

R Example – Finding Model-Free Policy (Continued)

How does one validate the model, as well as validate (and possibly update) the previously-determined optimal policy?

  • Sample a new set of data from environment
  • Determine optimal policy function again, with the same model
  • Then compare the new policy function with the previous policy function 
R Example – Validating Model-Free Policy

What are the state-value and state-action value functions?

  • Define the two value functions
  • Show how they impact policy evaluation and improvement
  • Illustrate with an R MDP example for moving a pawn 
Policy Evaluation and Iteration

How do MDP problem parameters affect the optimal policy solution?

  • Introduction to the discount factor, “gamma”
  • Show how gamma affects policy moving a pawn
  • Show how other parameters affect policy moving a pawn 
R Example – Moving a Pawn with Changed Parameters

How gamma affects policy improvement and optimal policy determination by diving deeper into the nature of the discount factor, gamma?

  • Explain how the discount factor determines the value function
  • Show how the value function determines policy
  • Present an R example of discount and rewards affecting policy 
Discount Factor and Policy Improvement

What is the nature of the Monte Carlo Model-Free approach to solving Reinforcement Learning problems?

  • Describe the characteristics of the Monte Carlo approach
  • Describe random versus epsilon-greedy action selection
  • Illustrate with an R race-to-goal example 
Monte Carlo Methods

What is the nature of the Model-Free Q-Learning approach to solve Reinforcement Learning problems?

  • Describe Q-Learning as an off-policy learning concept
  • Walk through the Q-Learning update rule
  • Illustrate Q-Learning with an R example 
Environment and Q-Learning Functions with R

Diving deeper into the nature of Q-Learning.             

Learning Episode and State-Action Functions in R

Explore the characteristics of the SARSA algorithm.

State-Action-Reward-State-Action (SARSA)

What is the nature of the Simulated Annealing algorithm alternative to Q-Learning?

  • Describe the characteristics of the Simulated Annealing approach
  • Describe probabilistic action selection derived from Boltzmann distribution metaheuristic
  • Illustrate with an R simulated annealing 2x2 grid example 
Simulated Annealing – An Alternative to Q-Learning

How does one incorporate the discount factor into the previous Model-Free Q-Learning Reinforcement Learning algorithm?

  • Modify the Q-Learning algorithm to include a discount factor
  • Include the aggregation of rewards by episode
  • Illustrate modified Q-Learning algorithm with R examples 
Q-Learning with a Discount Factor

How does one demonstrate the effects of Q-Learning algorithm control parameters using effective visualizations?

  • Use the popular R ggplot2 package to create visualizations
  • Examine effects of epsilon, alpha, and gamma control parameters
  • Create color-based line plots of Q-values and rewards 
Visual Q-Learning Examples
About the Instructor
Packt Publishing
3.9 Average rating
8,109 Reviews
58,370 Students
686 Courses
Tech Knowledge in Motion

Packt has been committed to developer learning since 2004. A lot has changed in software since then - but Packt has remained responsive to these changes, continuing to look forward at the trends and tools defining the way we work and live. And how to put them to work.

With an extensive library of content - more than 4000 books and video courses -Packt's mission is to help developers stay relevant in a rapidly changing world. From new web frameworks and programming languages, to cutting edge data analytics, and DevOps, Packt takes software professionals in every field to what's important to them now.

From skills that will help you to develop and future proof your career to immediate solutions to every day tech challenges, Packt is a go-to resource to make you a better, smarter developer.

Packt Udemy courses continue this tradition, bringing you comprehensive yet concise video courses straight from the experts.