Extra Fundamentals of R

Name: Extra Fundamentals of R
Rating: 4.2 (22 reviews)

Understanding R graphics, how to set up "real-world" simulations, and how to process non-numeric character and text in R

Created byGeoffrey Hubona, Ph.D.

Last updated 7/2020

English

English [Auto],

What you'll learn

Understand and use the base, lattice and ggplot graphics systems in R.
Be able to simulate many 'real-world' and practical "what-if" scenarios to determine likely outcomes.
Have a though understanding of, and ability to effectively use, the text and string variable processing capabilities in R.
Know how to use and implement R's text-based "regular expression" features and functions.

Course content

8 sections • 66 lectures • 12h 20m total length

Introduction1:35
Explore extra fundamentals of R, focusing on graphics systems such as base lavis and g.d. plot, programming in simulation, and processing text data.
Comparing Base and GGPlot2 Graphics15:15
One of the greatest strengths of the R language is surely the base graphics capabilities. Grid graphics, lattice, ggplot2, and the many R packages that interface with javascript D3graphics have added astounding capabilities, well beyond what can be achieved with base graphics alone. Nevertheless, the quick, one line, base graphics plots ( like plot() ) are a tremendous aid to data exploration, and are responsible for a good bit of the "flow" of an R session.
Continue Graphics Capabilities and Comparisons7:25
ggplot2 is a plotting system for R, based on the grammar of graphics, which tries to take the good parts of base and lattice graphics and none of the bad parts. It takes care of many of the fiddly details that make plotting a hassle (like drawing legends) as well as providing a powerful model of graphics that makes it easy to produce complex multi-layered graphics.
More Graphics Capabilities and Comparisons15:05
Graphical Parameters

You can customize many features of your graphs (fonts, colors, axes, titles) through graphic options.

One way is to specify these options in through the par( ) function. If you set parameter values here, the changes will be in effect for the rest of the session or until you change them again. The format ispar(optionname=value, optionname=value, ...)

# Set a graphical parameter using par() par() # view current settings opar <- par() # make a copy of current settings par(col.lab="red") # red x and y labels hist(mtcars$mpg) # create a plot with these new settings par(opar) # restore original settings

A second way to specify graphical parameters is by providing the optionname=value pairs directly to a high level plotting function. In this case, the options are only in effect for that specific graph.

# Set a graphical parameter within the plotting function hist(mtcars$mpg, col.lab="red")

See the help for a specific high level plotting function (e.g. plot, hist, boxplot) to determine which graphical parameters can be set this way.
Adding Text to Graphics7:09
Mathematical and Drawing Functions11:03

Fitting Non-Linear Curves in Base15:49
Explore fitting nonlinear curves with base graphics and lattice for multivariate displays, and visualize predicted versus actual fish recruitment using nonlinear fitting, lines, and interactive legend placement.
More Base Non-Linear Plots18:45
Explore non-linear data fitting in R using loess, GAM, and polynomial models to compare fits on deer jawbone age data, emphasizing window width, overfitting, and model selection.
Base Boxplots and Bargraphs14:32
Boxplots

Boxplots can be created for individual variables or for variables by group. The format is boxplot(x, data=), where x is a formula and data= denotes the data frame providing the data. An example of aformula is y~group where a separate boxplot for numeric variable y is generated for each value of group. Add varwidth=TRUE to make boxplot widths proportional to the square root of the samples sizes. Addhorizontal=TRUE to reverse the axis orientation.

# Boxplot of MPG by Car Cylinders

boxplot(mpg~cyl,data=mtcars, main="Car Milage Data", xlab="Number of Cylinders", ylab="Miles Per Gallon")
Introduction to Lattice Graphics13:30
Lattice Graphs

The lattice package, written by Deepayan Sarkar, attempts to improve on base R graphics by providing better defaults and the ability to easily display multivariate relationships. In particular, the package supports the creation of trellis graphs - graphs that display a variable or the relationship between variables, conditioned on one or more other variables.

The typical format is

graph_type(formula, data=)

where graph_type is selected from the listed below. formula specifies the variable(s) to display and any conditioning variables . For example ~x|A means display numeric variable x for each level of factor A.y~x | A*B means display the relationship between numeric variables y and x separately for every combination of factor A and B levels. ~x means display numeric variable x alone.
Superposition and Lattice Exercise4:26
Lattice Exercise Solution15:40

"In Living Color" Exercises Solution Explained15:45
Explore static graphics in R using lattice and ggplot, with multivariate visualizations, heat maps, and 3D surfaces like the Mobius strip, plus panel functions and color palettes.
Finish "In Living Color" Exercise and Begin Lattice Graphics15:18
Lattice Layouts, Groups, and Aspect Ratios14:12
Plotting the Titanic Data Set and Begin GGPlot Graphics11:54
Analyze Titanic survival by class and gender with bar charts, then use ggplot to arrange layouts and multi-plot panels for comparative visuals.
GGPlot: Non-Linear Fits and Plots15:31
Histograms, Bar Charts and Density Plots16:59
Explore histograms and density plots to compare diamond carat weight distributions and bandwidth effects. Visualize earthquake data with scatter plots, box plots, and a 3d cloud plot conditioned on magnitude.

Cuckoohost and Other Plots14:51
Demonstrate using R to load datasets and create plots that compare cuckoo and host egg lengths and breadths, connect paired observations, and explore tomato yield versus salinity with box plots.
Finish Cuckoohosts and Begin Simulation16:07
Explore comparing quantitative versus factor representations in R using box plots, variance concepts, and scale types, then apply Monte Carlo simulation to approximate exponential distribution parameters.
Simulating a Coin Tossing Game of Chance (part 1)14:10
Simulation uses methods based on random numbers to simulate a process of interest on the computer. The goal is to learn important statistical and/or practical information about the process. In statistics, simulations can be used to create simulated data sets in order to study the accuracy of mathematical approximations and the effect of assumptions being violated. We will study properties of some quantities that can be calculated from a set of data which are a random draw from a population. Random numbers form a basic tool for any simulation study. Simulations require the ability to generate random numbers. On a computer, it is only possible to generate 'pseudo-random' numbers which for practical purposes behave as if they were drawn randomly.
Simulating a Coin Tossing Game of Chance (part 2)14:05
Simulating the Return of Top-Hats to Rightful Owners (part 1)15:57
Finish Simulating Top-Hat Returns and Begin Collecting Baseball Cards for Profit11:51

Collecting Baseball Cards (part 1 continued)11:24
Collecting Baseball Cards (part 2)12:42
Use replicate to simulate collecting baseball cards in R with replacement, weighing 5-cent random pulls against 25-cent replacements, and identify an optimal purchase strategy to minimize cost.
Collecting Baseball Cards (part 3)10:30
Learn how to minimize the expected cost of baseball card purchases by simulating different purchase counts, implementing an R function to find the optimal number.
Collecting Quarters Exercise Solution6:06
Simulate purchasing 100 quarters to estimate how many unique states appear, assess the probability of at least 45, and compute the expected unique count and missing-quarters cost.
Streaky Baseball Behavior (part 1)11:52
Streaky Baseball Behavior (part 2)12:23
Sam and Annie Arrive at the Empire State Building (part 1)6:36
Explore parametric and non-parametric inference and bootstrapping for mean differences. Simulate the Sleepless in Seattle scenario to estimate the probability Annie arrives before Sam and the arrival time difference.
Hats and Streakiness Exercise7:30
Sam and Annie Arrive at the Empire State Building (part 2)11:14
Simulate a thousand uniform arrival times for Sam and Annie. Estimate the probability Annie arrives before Sam and the mean arrival difference with its standard error, using plots.

Checking Hats Exercise Solution6:43
More Streakiness Exercise Solution11:02
Examine a switch-based measure of streakiness in Utley's 2006 hitting sequence by counting binary switches and simulating permutations to show 60 switches are not extreme, not a good measure.
Standard Normal Monte Carlo Simulation6:39
Estimating Mean Squared Error of a Trimmed Mean8:03
Explore estimating the mean squared error of a trimmed mean in R by sorting data, trimming the extremes, and comparing the simulated mean to the true mean.
Estimating a Confidence Level11:37
Estimate a confidence level and a confidence interval for variance via Monte Carlo using chi-square upper limits; explore nonparametric, simulation-based coverage assessment.
Empirical Confidence Level4:57
Estimating the Taxi Population (part 1)9:11
Estimate the taxi population using two heuristics: the maximum observed and twice the sample mean, and compare their bias through simulation using a uniform distribution.
Estimating the Taxi Population (part 2)7:32
Permutation Tests (part 1)13:30
Permutation Tests (part 2)16:55
Discover how to perform a permutation two-sample test in R to compare unbalanced groups (soybean vs linseed weights), using 1000 replications to assess two-tailed significance, yielding a non-significant result.
The Bootstrap and Jackknife (part 1)9:32
The Bootstrap and Jackknife (part 2)11:37
Late to Class Again Exercise3:01
Use simulation in R to estimate five-day travel time, late probability after 30 minutes given normal distribution with mean 20 and sd 4, and longest travel time, with standard error.

Late to Class Exercise Solution9:59
Character and String Manipulation8:39
Handling and processing text strings in R? Wait a second . . . you exclaim, R is not a scripting language like Perl, Python, or Ruby. Why would you want to use R for handling and processing text? Well, because sooner or later (I would say sooner than later) you will have to deal with some kind of string manipulation for your data analysis. So it's better to be prepared for such tasks and know how to perform them inside the R environment.
Displaying and Concatenating Strings (part 1)10:15
Another very useful function is cat() which allows us to concatenate objects and print them either on screen or to a file. Its usage has the following structure:

cat(..., file = "", sep = " ", fill = FALSE, labels = NULL, append = FALSE)
Displaying and Concatenating Strings (part2)14:00
Manipulating Parts of a String10:00
Utilize the substring function to extract or replace parts of strings using first and last indices (last defaults to end), vectorized across string vectors, with replacement truncated to fit length.
Breaking Apart Character Values14:13
string split in R breaks a string into a list of words, shows counting words, flattening results with unlist, and applying to strings with regular expressions.
What are Regular Expressions? (slides)10:27
A regular expression (a.k.a. regex) is a special text string for describing a certain amount of text. This “certain amount of text” receives the formal name of pattern. Hence we say that a regular expression is a pattern that describes a set of strings.
Using Regular Expressions in R (part 1)13:31
There are two main aspects that we need to consider about regular expressions in R. One has to do with the functions designed for regex pattern matching. The other aspect has to do with the way regex patterns are expressed in R. In this section we are going to talk about the latter issue: the way R works with regular expressions. Some find more convenient to first cover the specificities of R around regex operations, before discussing the functions and how to interact with regex patterns.
Using Regular Expressions in R (part 2)11:36
Reversing a String Exercise1:44

Reverse String Exercise Solution9:29
Reverse any string in R by splitting into characters, unlisting, reversing with a for loop, and printing, using a function with a default argument.
The Regexpr() and Gregexpr() Functions (part 1)12:41
To find exactly where the pattern is found in a given string, we can use the regexpr() function. This function returns more detailed information than grep() providing us: a) which elements of the text vector actually contain the regex pattern, and b) identifies the position of the substring that is matched by the regular expression pattern.

# some text

text = c("one word", "a sentence", "you and me", "three two one")

# default usage

regexpr("one", text)
The Regexpr() and Gregexpr() Functions (part 2)9:48
The function gregexpr() does practically the same thing as regexpr(): identify where a pattern is within a string vector, by searching each element separately. The only difference is that gregexpr() has an output in the form of a list. In other words, gregexpr() returns a list of the same length as text, each element of which is of the same form as the return value for regexpr(), except that the starting positions of every (disjoint) match are given.

# some text

text = c("one word", "a sentence", "you and me", "three two one")

# pattern

pat = "one"

# default usage

gregexpr(pat, text)
Testing a Filename for a Suffix8:40
Test string suffixes in R using a user-defined function with string split, fixed = TRUE, to split on periods and return true or false for matches.
Forming Filenames8:46
Generate five histograms of 100 random normal variates with increasing standard deviations. Save five PTF files named q1 through q5 by pasting filenames with no spaces to the default directory.
Substituting Text and Tagging Text14:54
Explore substitutions and tagging with regular expressions to modify text, convert currency strings to numeric, and extract values using sub and gsub in R's text processing toolkit.
Finding Words in Text Passages14:45
grep() is perhaps the most basic functions that allows us to match a pattern in a string vector. The first argument in grep() is a regular expression that specifies the pattern to match. The second argument is a character vector with the text strings on which to search. The output is the indices of the elements of the text vector for which there is a match. If no matches are found, the output is an empty integer vector.

# some text

text = c("one word", "a sentence", "you and me", "three two one")

# pattern

pat = "one"

# default usage

grep(pat, text)
Manipulating the Component Names of List Structures11:07
Explore how to manipulate component names in R lists, use names and unlist, understand vector versus list behavior, and apply lapply and sapply for list processing.
Sorting and Ordering Words12:21
Determining and Plotting Word Frequency6:33

Requirements

Students will need to install the no-cost R console and the no-cost RStudio application (instructions and provided).

Description

Extra Fundamentals of R is an extension of the Udemy course Essential Fundamentals of R. Extra Fundamentals of R introduces additional topics of interest and relevance utilizing many specific R-scripted examples. These broad topics include:

(1) Details on using Base, GGPlot and Lattice graphics;

(2) An introduction to programming and simulation in R; and

(3) Character and string processing in R.

All materials, scripts, slides, documentation and anything used or viewed in any one of the video lessons is provided with the course. The course is useful for both R-novices, as well as to intermediate R users. Rather than focus on specific and narrow R-supported skill sets, the course paints a broad canvas illustrating many specific examples in three domains that any R user would find useful. The course is a natural extension of the more basic Udemy course, Essential Fundamentals of R and is highly recommended for those students, as well as for other new students (and for practicing professionals) interested in the three domains enumerated above.

Base, GGplot and Lattice (or "trellis) graphics are the three principal graphics systems in R. They each operate under different "rules" and each present useful and often brilliant graphics displays. However, each of these three graphics systems are generally designed and used for different domains or applications.

There are many different programming and simulation scenarios that can be modeled with R. This course provides a good sense for some of the potential simulation applications through the presentation of 'down-to-earth,' practical domains or tasks that are supported. The examples are based on common and interesting 'real-world' tasks: (1) simulating a game of coin-tossing; (2) returning Top-Hats checked into a restaurant to their rightful owners; (3) collecting baseball cards and state quarters for profit: (4) validating whether so-called "streaky" behavior, such as have a string of good-hitting behavior in consecutive baseball games, is really unusual from a statistical point of view; (4) estimating the number of taxicabs in a newly-visited city; and (5) estimating arrival times for Sam and Annie at the Empire State Building ("Sleepless in Seattle").

R is likely best known for the ability to process numerical data, but R also has quite extensive capabilities to process non-quantitative text (or character) and string variables. R also has very good facilities for implementing powerful "regular expression" natural-language functions. An R user is bested served with an understanding of how these text (or character) and string processing capabilities "work."

Most sessions present "hands-on" material that make use of many extended examples of R functions, applications, and packages for a variety of common purposes. RStudio, a popular, open source Integrated Development Environment (IDE) for developing and using R applications, is utilized in the program, supplemented with R-based direct scripts (e.g. 'command-line prompts') when necessary.

Who this course is for:

Any novice or intermediate R user would benefit from this course.
Appropriate candidate students for this course include undergraduate and graduate students, college and university faculty, and practicing professionals, particularly in quantitative or analytics fields.
It is useful to have some rudimentary exposure to using R in a sample session, executing R script.

Extra Fundamentals of R

What you'll learn

Explore related topics

Course content

Base and GGPlot2 Graphics in R6 lectures • 58min

Finish Base Graphics Capabilities, Begin Lattice Graphics6 lectures • 1hr 23min

Lattice and GGPlot Graphics6 lectures • 1hr 30min

Programming and Simulation 16 lectures • 1hr 27min

Programming and Simulation 29 lectures • 1hr 30min

Programming and Simulation 313 lectures • 2hr

Character Manipulation and String Processing10 lectures • 1hr 44min

More Text and String Processing10 lectures • 1hr 49min

Requirements

Description

Who this course is for: