R: Complete Machine Learning Solutions

Reading a dataset is the first and foremost step in data exploration. We need to learn to how to do that

• Download the train dataset
• Use read.csv and the str function to load and display the dataset respectively

In R, since nominal, ordinal, interval, and ratio variable are treated differently in statistical modeling, we have to convert a nominal variable from a character into a factor.

• Display the structure of the data using str
• Find the attribute name, data type, and values contained in each attribute
• Use the factor function to transform data from character to factor

Missing values affect the inference of a dataset. Thus it is important to detect them.

• Sum up all the NA values 
• Divide the sum by the number of values in each attribute
• Apply the calculation to all attributes using sapply

After detecting missing values, we need to impute them as their absence may affect the conclusion.

• Produce statistics using a table
• Sort the table. Use str_match to find the title with missing values
• Assign the missing value with the mean value

After imputing the missing values, you should perform an exploratory analysis to summarize the data characteristics.

• Generate a bar plot and histogram of each attribute
• Examine the relation between all attributes, two at a time

The exploratory analysis helps users gain insights into how single or multiple variables may affect the survival rate. However, it does not determine what combinations may generate a prediction model. We need to use a decision tree for that. 

• Construct a data split function
• Split data according to the need
• Generate the prediction model and plot the tree

After constructing the prediction model, it is important to validate how the model performs while predicting the labels.

• Predict the survival of the testing set
• Generate the statistics of the output matrix using a confusion matrix

Univariate statistics deals with a single variable and hence is very simple.

• Load data into a data frame. Compute the length of the variable.
• Obtain mean, median, standard deviation and variance.
• Obtain IQR, quantile, maxima, minima, and so on. Plot a histogram.

To analyze the relation among more than two variables, multivariate analysis is done. 

• Get the co-variance matrix
• Obtain the correlation matrix

Assessing the relation between dependent and independent variables is carried out through linear regression.

• Fit variables into a model
• Create an analysis of a variance table
• Plot the regression line

To validate that the experiment results are significant, hypothesis testing is done. 

• Conduct an exact binomial test

To compare means of two different groups, one- and two-sample t-tests are conducted.

• Visualize the attributes
• Perform the statistical procedure

Comparing a sample with a reference probability or comparing cumulative distributions of two data sets calls for a Kolmogorov- Smirnov test.

• Check a normal distribution with one sample Kolmogrov-Smirnov test.
• Generate uniformly distributed sample data. 
• Plot the ecdf of two samples. Apply a two-sample Kolmogrov-Smirnov test.

The Wilcoxon Test is a non-parametric test for null hypothesis.

• Plot data with boxplot
• Perform a Wilcoxon Rank Sum test

To check the distribution of categorical variables of two groups, Pearson’s chi-squared test is used. 

• Use the contingency table to make the counting table
• Plot the mosaic plot
• Perform Pearson’s Chi-squared test.

To examine the relation between categorical independent variables and continuous dependent variables, Anova is used. When there is a single variable, one-way ANOVA is used. 

• Visualize the data with a boxplot
• Conduct a one-way ANOVA and perform ANOVA analysis
• Plot the differences in mean level.

When there are two categorical values to be compared, two-way ANOVA is used.

• Plot the two boxplots.
• Use an interaction plot.
• Perform two-way ANOVA. Plot the differences in mean level. 

Linear regression is the simplest model in regression and can be used when there is one predictor value. 

• Prepare data with a linear relationship between predictor and response variables
• Generate the regression line
• Plot the regression line

To obtain summarized information of a fitted model, we need to learn how to summarize linear model fits. 

• Compute the summary using summary function

It would be really convenient for us if we could predict unknown values. You can do that using linear regression. 

• Build a linear fitted model
• Compute the prediction result using confidence interval
• Compute the prediction result using prediction interval

To check if the fitted model adequately represents the data, we perform diagnostics.

• Generate a diagnostic plot

In the case of a non-linear relationship between predictor and response variables, a polynomial regression model is formed. We need to fit the model. This video will enable you to do that.

• Illustrate the polynomial regression model in formula

An outlier will cause diversion from the slope of the regression line. In order to avoid that, we need to fit a robust linear regression model. 

• Generate the scatter plot
• Apply the rlm function
• Visualize the fitted line 

We will perform linear regression on a real-life example, the SLID dataset. 

• Load the SLID data. Fit all attributes.
• Generate the diagnostic plot.
• Test multi-colinearity and heteroscedasticity.

GLM generalizes linear regression by allowing the linear model to be related to the response variable via a link function and by allowing the magnitude of the variance of each measurement to be a function of its predicted value. 

• Input the independent and dependent variables
• Fit variables to a model
• Compare the fitted models with ANOVA function

GLM allows response variables with error distribution other than a normal distribution. We apply the Poisson model to see how that is done.

• Load a sample count data
• Apply the glm function
• View the fitted log-linear model

When a variable is binary, we apply the binomial model.

• Load the binary dependent variable
• Fit the model into the binary data
• Obtain a description using summary

GAM has the ability to deal with non-linear relationships between dependent and independent variables. We learn to fit a regression using GAM.

• Load the Boston dataset
• Generate a fitted model
• Summarize the GAM fit

Visualizing a GAM helps it to understand better.

• Generate a scatter plot
• Add regression line
• Visualize the fitted regression lines

You can also diagnose a GAM model to analyze it. 

• Produce the smoothing parameter estimation convergence information
• View the four diagnostic plots

Training and testing datasets are both essential for building a classification model.

• Preprocess the dataset. Remove attributes which are unimportant
• Split the data into training and testing sets
• Generate a sequence accordingly and interpret the output

A partitioning tree works on the basis of split condition starting from the base node to the terminal node. 

• Load the rpart package. Build a classification model
• Display the tree node details
• Generate the information graphic

Plotting the classification tree will make analyzing the data easier. You will learn to do this now.

• Plot the classification tree
• Specify parameters to adjust the layout

Before making a prediction, it is essential to compute the prediction performance of the model.

• Generate a predicted label and classification table for the testing dataset
• Generate a confusion matrix

There can be parts in a dataset which are not essential for classification. In order to remove these parts, we have to prune the dataset.

• Locate the record with minimum cross validation errors
• Extract the CP of the record and assign the value to churn
• Prune the classification tree

Conditional inference trees are better than traditional classification trees because they adapt the test procedures for selecting the output.

• Build the classification model
• Examine the built tree model

Visualizing a conditional inference tree will make it easier to extract and analyze data from the dataset.

• Plot the built classification model

Like the prediction performance of a traditional classification tree, we can also evaluate the performance of a conditional inference tree.

• Predict the category of the testing dataset
• Generate a classification table
• Determine the performance measurements

K-nearest neighbor classifier is a non parametric lazy learning method. Thus it has the advantages of both the types of methods.

• Build a classification model
• Generate a classification table. Generate a confusion matrix from it
• Examine the sensitivity and specificity

Classification in logistic regression is done based one or more features. It is more robust and doesn’t have as many conditions as the traditional classification model.

• Generate a logistic regression model. Generate the model’s summary
• Predict the categorical dependent variable of the testing dataset
• Generate the confusion matrix

The Naïve Bayes classifier is based on applying Bayes’ theorem with a strong independent assumption. 

• Specify the variables as first input parameters and churn label as the second input parameter in the function call
• Assign the classification model to the classifier variable
• Use a confusion matrix to calculate the performance measurement

Support vector machines are better at classification because they can capture complex relations between data points and provide both linear and non-linear classifications.

• Train a support vector machine
• Use different functions and arguments as desired for the output
• Obtain a summary of the built support vector machine

To control our training errors and margins, we use the cost function. The SVM classifier is affected by the cost. 

• Create an iris subset
• Use SVM with small cost and large cost and see its effect

To visualize the SVM fit, we can use the plot function.

• Train SVM and and use plot to visualize the fitted SVM
• Specify appropriate parameters while generating the scatter plot

We can use the trained SVM to predict labels on a model.

• Obtain the predicted labels of the testing dataset.
• Generate the classification table. Compute co-efficients of a contingency table.
• Use a confusion matrix to measure performance.

According to the desired output, you may need to generate different combinations of gamma and cost to train different SVMs. This is called tuning.

• Generate a set of parameters.
• Obtain the best parameters. Train a new SVM.
• Obtain a classification table. Compare the two models.

A neural network is used in classification, clustering and prediction. Its efficiency depends on how well you train it. Let’s learn to do that.

• Split the dataset into training and testing datasets.
• Add the required columns. Train the network model.
• Configure the hidden neurons. Examine the information of the neural network model.

Visualizing a neural network would make understanding the process easier for you. 

• Visualize the trained neural network using plot.
• View the generalized weights using gwplot.

Similar to other classification models, we can predict labels using neural networks and also validate performance using confusion matrix.

• Create an output probability matrix. Convert the probability matrix to class labels.
• Generate a classification matrix based on the labels obtained.
• Employ a confusion matrix to measure the prediction performance of the built neural network.

Nnet provides the functionality  to train feed-forward neural networks with backpropagation.

• Use nnet to train the neural network. Set different parameters in the function.
• Use summary function to obtain information about the built neural network.

As we have already trained the neural network using nnet, we can use the model to predict labels.

• Generate the predicted labels based on a testing dataset.
• Generate a classification table based on predicted labels.
• Employ a confusion matrix to measure the prediction performance of the trained neural network.

The k-fold cross-validation technique is a common technique used to estimate the performance of a classifier as it overcomes the problem of over-fitting. In this video we will illustrate how to perform a k-fold cross-validation:

• Generate an index with 10 folds with the cut function
• Use a for loop to perform a 10-fold cross-validation
• Generate average accuracies with the mean function

In this video, we will illustrate how to use tune.svm to perform 10-fold cross-validation and obtain the optimum classification model.

• Apply tune.svm to the training dataset
• Obtain the summary information of the model
• Access the performance details of the tuned model
• Generate a classification table

In this video we will demonstrate how to perform k-fold cross validation using the caret package.

• Set up the control parameter
• Train the classification model on telecom churn data
• Examine the output of the generated model

This video will show you how to rank the variable importance with the caret package.

• Estimate the variable importance
• Generate the variable importance plot

In this video we will show how to find highly correlated features using the caret package.

• Remove the features that are not coded in numeric characters
• Obtain the correlation of each attribute
• Obtain the names of highly correlated attributes

To measure the performance of a regression model, we can calculate the distance from the predicted output and the actual output as a quantifier of the performance of the model. In this video we will illustrate how to compute these measurements from a built regression model.

• Load the dataset
• Calculate the root mean square error, relative square error and R-Square value

In this video we will demonstrate how to retrieve a confusion matrix using the caret package

• Train an svm model using the training dataset
• Predict labels using the fitted model
• Generate a classification table and a confusion matrix

In this video, we will demonstrate how to illustrate an ROC curve and calculate the AUC to measure the performance of a classification model.

• Install and load the ROCR package
• Visualize the ROC curve using the plot function

In this video we will use the function provided by the caret package to compare different algorithm-trained models on the same dataset.

• Install and load the pROC library
• Generate the ROC curve of each model, and plot the curve

In this video we will see how to measure performance differences between fitted models with the caret package.

• Resample the three generated models and obtain its summary
• Plot the re-sampling result in the ROC metric or box-whisker plot

The adabag package implements both boosting and bagging methods. For the bagging method, the package first generates multiple versions of classifiers, and then obtains an aggregated classifier. Let’s learn the bagging method from adabag to generate a classification model.

• Install the adabag package and use the bagging function
• Generate the classification model
• Obtain a classification table and average error

To assess the prediction power of a classifier, you can run a cross validation method to test the robustness of the classification model. This video will show how to use bagging.cv to perform cross validation with the bagging method.

• Use bagging.cv to perform cross-validation 
• Obtain the confusion matrix
• Retrieve the minimum estimation error

Boosting starts with a simple or weak classifier and gradually improves it by reweighting the misclassified samples. Thus, the new classifier can learn from previous classifiers. One can use the boosting method to perform ensemble learning. Let’s see how to use the boosting method to classify the telecom churn dataset.

• Use the boosting function from the adabag package
• Make a prediction based on the boosted model and testing dataset
• Retrieve the classification table and obtain average errors

Similar to the bagging function, adabag provides a cross validation function for the boosting method, named boosting.cv. In this video, we will learn how to perform cross-validation using boosting.cv.

• Use boosting.cv to cross-validate the training dataset
• Obtain the confusion matrix
• Retrieve the average errors

Gradient boosting creates a new base learner that maximally correlates with the negative gradient of the loss function. One may apply this method on either regression or classification problems. But first, we need to learn how to use gbm.

• Install the gbm package and use the gbm function to train a training dataset
• Use cross-validation and plot the ROC curve
• Use the coords function and obtain a classification table from the predicted results

A margin is a measure of certainty of a classification. It calculates the difference between the support of a correct class and the maximum support of an incorrect class. This video will show us how to calculate the margins of the generated classifiers.

• Use the margins function
• Use the plot function to plot a marginal cumulative distribution graph
• Compute the percentage of negative margin

The adabag package provides the errorevol function for a user to estimate the ensemble method errors in accordance with the number of iterations. Let’s explore how to use errorevol to show the evolution of errors of each ensemble classifier.

• Use the errorevol function for error evolution of boosting classifiers
• Use the errorevol function for error evolution of bagging classifiers

Random forest grows multiple decision trees which will output their own prediction results. The forest will use the voting mechanism to select the most voted class as the prediction result. In this video, we illustrate how to classify data using the randomForest package.

• Install and load the randomForest package 
• Plot the mean square error of the forest object
• Use the varImpPlot function, the margin function, hist, and  boxplot

At the beginning of this section, we discussed why we use ensemble learning and how it can improve the prediction performance. Let’s now validate whether the ensemble model performs better than a single decision tree by comparing the performance of each method.

• Estimate the error rate of the bagging model
• Estimate the error rate of the boosting method
• Estimate the error rate of the random forest model
• Use churn.predict and estimate the error rate of single decision tree

Hierarchical clustering adopts either an agglomerative or a divisive method to build a hierarchy of clusters. This video shows us how to cluster data with the help of hierarchical clustering.

• Load the data and save it 
• Examine the dataset structure
• Use agglomerative hierarchical clustering to cluster data

In this video we demonstrate how to use the cutree function to separate the data into a given number of clusters.

• Categorize the data and examine its cluster labels
• Count the number of data within each cluster
• Visualize how data is clustered

In this video, we will demonstrate how to perform k-means clustering on the customer dataset.

• Use k-means to cluster the data
• Inspect the center of each cluster
• Draw a scatter plot of data and color the points

We will now illustrate how to create a bivariate cluster plot.

• Install and load the cluster package
• Draw a bivariate cluster plot

In this video we will see how to compare different clustering methods using cluster.stat from the fpc package.

• Install and load the fpc package
• Use different clustering methods
• Generate the cluster statistics of each clustering method

In this video we will see how to compute silhouette information.

• Use k-means to generate a k-means object
• Compute and plot the silhouette information

In this video we will discuss how to find the optimum number of clusters for the k-means clustering method.

• Calculate the withinss of different numbers of clusters and plot them
• Calculate the average silhouette and plot it

In this video, we will demonstrate how to use DBSCAN to perform density-based clustering.

• Install and load the fpc and mlbench packages
• Cluster data with regard to its density measurement

In this video, we will demonstrate how to use the model-based method to determine the most likely number of clusters.

• Install and load the mclust library
• Perform model-based clustering on the customer dataset

A dissimilarity matrix can be used as a measurement for the quality of a cluster. In this video, we will discuss some techniques that are useful to visualize a dissimilarity matrix.

• Install and load the seriation package
• Visualize the dissimilarity matrix

In this video, we will demonstrate how clustering methods differ with regard to data with known clusters.

• Install and load the package png
• Perform k-means and the dbscan clustering method on the handwriting digits

Before starting with a mining association rule, you need to transform the data into transactions. This video will show how to transform any of a list, matrix, or data frame into transactions.

• Install and load the arule package
• Use the as function
• Transform the matrix-format data and data-frame-format dataset into transactions

The arule package uses its own transactions class to store transaction data. As such, we must use the generic function provided by arule to display transactions and association rules. Let’s see how to display transactions and association rules via various functions in the arule package.

• Obtain a LIST representation and use the summary function
• Use the inspect function and filter transactions by size
• Use the image function and itemFrequenctPlot

Association mining is a technique that can discover interesting relationships hidden in transaction datasets. This approach first finds all frequent itemsets and then generates strong association rules from frequent itemsets. In this video, we see how to perform association analysis using the apriori rule.

• Load the Groceries dataset and examine the summary
• Use itemFrequencyPlot and apriori
• Inspect the first few rules

Among the generated rules, we sometimes find repeated or redundant rules (for example, one rule is the subset of another rule). Let’s explore how to prune (or remove) repeated or redundant rules.

• Find redundant rules
• Remove redundant rules

Besides listing rules as text, you can visualize association rules, making it easier to find the relationship between itemsets. In this video, we will learn how to use the aruleViz package to visualize the association rules.

• Install and load the arulesViz package 
• Make a scatter plot from the pruned rules and add jitter to it
• Plot soda_rule in a graph plot and a ballon plot

An apriori algorithm performs a breadth-first search to scan the database. So, support counting becomes time consuming. Alternatively, if the database fits into the memory, you can use the Eclat algorithm, which performs a depth-first search to count the supports. Let’s see how to use the Eclat algorithm.

• Use the eclat function to generate a frequent itemset
• Obtain the summary information
• Examine the top ten support frequent itemsets

In addition to mining interesting associations within the transaction database, we can mine interesting sequential patterns using transactions with temporal information. This video demonstrates how to create transactions with temporal information.

• Install and load the arulesSequences package 
• Turn the list into transactions and use the inspect function
• Obtain summary information and read transaction data in basket format

In contrast to association mining, we should explore patterns shared among transactions where a set of itemsets occurs sequentially. One of the most famous frequent sequential pattern mining algorithms is the Sequential Pattern Discovery using Equivalence classes (SPADE) algorithm. Let’s see how to use SPADE to mine frequent sequential patterns.

• Use the cspade function to generate frequent sequential patterns
• Examine the summary of the frequent sequential patterns
• Transform a generated sequence format data back to the data frame