Are you interested in understanding machine learning concepts and building real-time projects with R, but don’t know where to start? Then, this is the perfect course for you!
The aim of machine learning is to uncover hidden patterns, unknown correlations, and find useful information from data. In addition to this, through incorporation with data analysis, machine learning can be used to perform predictive analysis. With machine learning, the analysis of business operations and processes is not limited to human scale thinking; machine scale analysis enables businesses to capture hidden values in big data.
Machine learning has similarities to the human reasoning process. Unlike traditional analysis, the generated model cannot evolve as data is accumulated. Machine learning can learn from the data that is processed and analyzed. In other words, the more data that is processed, the more it can learn.
R, as a dialect of GNU-S, is a powerful statistical language that can be used to manipulate and analyze data. Additionally, R provides many machine learning packages and visualization functions, which enable users to analyze data on the fly. Most importantly, R is open source and free.
Using R greatly simplifies machine learning. All you need to know is how each algorithm can solve your problem, and then you can simply use a written package to quickly generate prediction models on data with a few command lines.
By taking this course, you will gain a detailed and practical knowledge of R and machine learning concepts to build complex machine learning models.
What details do you cover in this course?
We start off with basic R operations, reading data into R, manipulating data, forming simple statistics for visualizing data. We will then walk through the processes of transforming, analyzing, and visualizing the RMS Titanic data. You will also learn how to perform descriptive statistics.
This course will teach you to use regression models. We will then see how to fit data in tree-based classifier, Naive Bayes classifier, and so on.
We then move on to introducing powerful classification networks, neural networks, and support vector machines. During this journey, we will introduce the power of ensemble learners to produce better classification and regression results.
We will see how to apply the clustering technique to segment customers and further compare differences between each clustering method.
We will discover associated terms and underline frequent patterns from transaction data.
We will go through the process of compressing and restoring images, using the dimension reduction approach and R Hadoop, starting from setting up the environment to actual big data processing and machine learning on big data.
By the end of this course, we will build our own project in the e-commerce domain.
This course will take you from the very basics of R to creating insightful machine learning models with R.
We have combined the best of the following Packt products:
The source content have been received well by the audience. Here is a one of the reviews:
"good product, I enjoyed it"
- Ertugrul Bayindir
Meet your expert instructors:
Yu-Wei, Chiu (David Chiu) is the founder of LargitData a startup company that mainly focuses on providing big data and machine learning products. He has previously worked for Trend Micro as a software engineer, where he was responsible for building big data platforms for business intelligence and customer relationship management systems.
Dipanjan Sarkar is an IT engineer at Intel, the world's largest silicon company, where he works on analytics, business intelligence, and application development. His areas of specialization includes software engineering, data science, machine learning, and text analytics.
Raghav Bali has a master's degree (gold medalist) in IT from the International Institute of Information Technology, Bangalore. He is an IT engineer at Intel, the world's largest silicon company, where he works on analytics, business intelligence, and application development.
Meet your managing editor:
This course has been planned and designed for you by me, Tanmayee Patil. I'm here to help you be successful every step of the way, and get maximum value out of your course purchase. If you have any questions along the way, you can reach out to me and our author group via the instructor contact feature on Udemy.
R must be first installed on your system to work on it
RStudio makes the process of development with R easier.
R packages are an essential part of R as they are required in all our programs. Let’s learn to do that.
You must know how to give data to R to work with data. You will learn that here.
Data manipulation is time consuming and hence needs to be done with the help of built-in R functions.
R is widely used for statistical applications. Hence it is necessary to learn about the built in functions of R.
To communicate information effectively and make data easier to comprehend we need graphical representation. You will learn to plot figures in this section.
Because of some limitations, it is a good practice to get data from external repositories. You will be able to do just that after this video.
Reading a dataset is the first and foremost step in data exploration. We need to learn to how to do that
In R, since nominal, ordinal, interval, and ratio variable are treated differently in statistical modeling, we have to convert a nominal variable from a character into a factor.
Missing values affect the inference of a dataset. Thus it is important to detect them.
After detecting missing values, we need to impute them as their absence may affect the conclusion.
After imputing the missing values, you should perform an exploratory analysis to summarize the data characteristics.
The exploratory analysis helps users gain insights into how single or multiple variables may affect the survival rate. However, it does not determine what combinations may generate a prediction model. We need to use a decision tree for that.
After constructing the prediction model, it is important to validate how the model performs while predicting the labels.
Univariate statistics deals with a single variable and hence is very simple.
To analyze the relation among more than two variables, multivariate analysis is done.
Assessing the relation between dependent and independent variables is carried out through linear regression.
To validate that the experiment results are significant, hypothesis testing is done.
To compare means of two different groups, one- and two-sample t-tests are conducted.
Comparing a sample with a reference probability or comparing cumulative distributions of two data sets calls for a Kolmogorov- Smirnov test.
The Wilcoxon Test is a non-parametric test for null hypothesis.
To check the distribution of categorical variables of two groups, Pearson’s chi-squared test is used.
To examine the relation between categorical independent variables and continuous dependent variables, Anova is used. When there is a single variable, one-way ANOVA is used.
When there are two categorical values to be compared, two-way ANOVA is used.
Linear regression is the simplest model in regression and can be used when there is one predictor value.
To obtain summarized information of a fitted model, we need to learn how to summarize linear model fits.
It would be really convenient for us if we could predict unknown values. You can do that using linear regression.
To check if the fitted model adequately represents the data, we perform diagnostics.
In the case of a non-linear relationship between predictor and response variables, a polynomial regression model is formed. We need to fit the model. This video will enable you to do that.
An outlier will cause diversion from the slope of the regression line. In order to avoid that, we need to fit a robust linear regression model.
We will perform linear regression on a real-life example, the SLID dataset.
GLM generalizes linear regression by allowing the linear model to be related to the response variable via a link function and by allowing the magnitude of the variance of each measurement to be a function of its predicted value.
GLM allows response variables with error distribution other than a normal distribution. We apply the Poisson model to see how that is done.
When a variable is binary, we apply the binomial model.
GAM has the ability to deal with non-linear relationships between dependent and independent variables. We learn to fit a regression using GAM.
Visualizing a GAM helps it to understand better.
You can also diagnose a GAM model to analyze it.
Training and testing datasets are both essential for building a classification model.
A partitioning tree works on the basis of split condition starting from the base node to the terminal node.
Plotting the classification tree will make analyzing the data easier. You will learn to do this now.
Before making a prediction, it is essential to compute the prediction performance of the model.
There can be parts in a dataset which are not essential for classification. In order to remove these parts, we have to prune the dataset.
Conditional inference trees are better than traditional classification trees because they adapt the test procedures for selecting the output.
Visualizing a conditional inference tree will make it easier to extract and analyze data from the dataset.
Like the prediction performance of a traditional classification tree, we can also evaluate the performance of a conditional inference tree.
K-nearest neighbor classifier is a non parametric lazy learning method. Thus it has the advantages of both the types of methods.
Classification in logistic regression is done based one or more features. It is more robust and doesn’t have as many conditions as the traditional classification model.
The Naïve Bayes classifier is based on applying Bayes’ theorem with a strong independent assumption.
Support vector machines are better at classification because they can capture complex relations between data points and provide both linear and non-linear classifications.
To control our training errors and margins, we use the cost function. The SVM classifier is affected by the cost.
To visualize the SVM fit, we can use the plot function.
We can use the trained SVM to predict labels on a model.
According to the desired output, you may need to generate different combinations of gamma and cost to train different SVMs. This is called tuning.
A neural network is used in classification, clustering and prediction. Its efficiency depends on how well you train it. Let’s learn to do that.
Visualizing a neural network would make understanding the process easier for you.
Similar to other classification models, we can predict labels using neural networks and also validate performance using confusion matrix.
Nnet provides the functionality to train feed-forward neural networks with backpropagation.
As we have already trained the neural network using nnet, we can use the model to predict labels.
The k-fold cross-validation technique is a common technique used to estimate the performance of a classifier as it overcomes the problem of over-fitting. In this video we will illustrate how to perform a k-fold cross-validation:
In this video, we will illustrate how to use tune.svm to perform 10-fold cross-validation and obtain the optimum classification model.
In this video we will demonstrate how to perform k-fold cross validation using the caret package.
This video will show you how to rank the variable importance with the caret package.
In this video we will show how to find highly correlated features using the caret package.
To measure the performance of a regression model, we can calculate the distance from the predicted output and the actual output as a quantifier of the performance of the model. In this video we will illustrate how to compute these measurements from a built regression model.
In this video we will demonstrate how to retrieve a confusion matrix using the caret package
In this video, we will demonstrate how to illustrate an ROC curve and calculate the AUC to measure the performance of a classification model.
In this video we will use the function provided by the caret package to compare different algorithm-trained models on the same dataset.
In this video we will see how to measure performance differences between fitted models with the caret package.
The adabag package implements both boosting and bagging methods. For the bagging method, the package first generates multiple versions of classifiers, and then obtains an aggregated classifier. Let’s learn the bagging method from adabag to generate a classification model.
To assess the prediction power of a classifier, you can run a cross validation method to test the robustness of the classification model. This video will show how to use bagging.cv to perform cross validation with the bagging method.
Boosting starts with a simple or weak classifier and gradually improves it by reweighting the misclassified samples. Thus, the new classifier can learn from previous classifiers. One can use the boosting method to perform ensemble learning. Let’s see how to use the boosting method to classify the telecom churn dataset.
Similar to the bagging function, adabag provides a cross validation function for the boosting method, named boosting.cv. In this video, we will learn how to perform cross-validation using boosting.cv.
Gradient boosting creates a new base learner that maximally correlates with the negative gradient of the loss function. One may apply this method on either regression or classification problems. But first, we need to learn how to use gbm.
A margin is a measure of certainty of a classification. It calculates the difference between the support of a correct class and the maximum support of an incorrect class. This video will show us how to calculate the margins of the generated classifiers.
The adabag package provides the errorevol function for a user to estimate the ensemble method errors in accordance with the number of iterations. Let’s explore how to use errorevol to show the evolution of errors of each ensemble classifier.
Random forest grows multiple decision trees which will output their own prediction results. The forest will use the voting mechanism to select the most voted class as the prediction result. In this video, we illustrate how to classify data using the randomForest package.
At the beginning of this section, we discussed why we use ensemble learning and how it can improve the prediction performance. Let’s now validate whether the ensemble model performs better than a single decision tree by comparing the performance of each method.
Hierarchical clustering adopts either an agglomerative or a divisive method to build a hierarchy of clusters. This video shows us how to cluster data with the help of hierarchical clustering.
In this video we demonstrate how to use the cutree function to separate the data into a given number of clusters.
In this video, we will demonstrate how to perform k-means clustering on the customer dataset.
We will now illustrate how to create a bivariate cluster plot.
In this video we will see how to compare different clustering methods using cluster.stat from the fpc package.
In this video we will see how to compute silhouette information.
In this video we will discuss how to find the optimum number of clusters for the k-means clustering method.
In this video, we will demonstrate how to use DBSCAN to perform density-based clustering.
In this video, we will demonstrate how to use the model-based method to determine the most likely number of clusters.
A dissimilarity matrix can be used as a measurement for the quality of a cluster. In this video, we will discuss some techniques that are useful to visualize a dissimilarity matrix.
In this video, we will demonstrate how clustering methods differ with regard to data with known clusters.
Before starting with a mining association rule, you need to transform the data into transactions. This video will show how to transform any of a list, matrix, or data frame into transactions.
The arule package uses its own transactions class to store transaction data. As such, we must use the generic function provided by arule to display transactions and association rules. Let’s see how to display transactions and association rules via various functions in the arule package.
Association mining is a technique that can discover interesting relationships hidden in transaction datasets. This approach first finds all frequent itemsets and then generates strong association rules from frequent itemsets. In this video, we see how to perform association analysis using the apriori rule.
Among the generated rules, we sometimes find repeated or redundant rules (for example, one rule is the subset of another rule). Let’s explore how to prune (or remove) repeated or redundant rules.
Besides listing rules as text, you can visualize association rules, making it easier to find the relationship between itemsets. In this video, we will learn how to use the aruleViz package to visualize the association rules.
An apriori algorithm performs a breadth-first search to scan the database. So, support counting becomes time consuming. Alternatively, if the database fits into the memory, you can use the Eclat algorithm, which performs a depth-first search to count the supports. Let’s see how to use the Eclat algorithm.
In addition to mining interesting associations within the transaction database, we can mine interesting sequential patterns using transactions with temporal information. This video demonstrates how to create transactions with temporal information.
In contrast to association mining, we should explore patterns shared among transactions where a set of itemsets occurs sequentially. One of the most famous frequent sequential pattern mining algorithms is the Sequential Pattern Discovery using Equivalence classes (SPADE) algorithm. Let’s see how to use SPADE to mine frequent sequential patterns.
Packt has been committed to developer learning since 2004. A lot has changed in software since then - but Packt has remained responsive to these changes, continuing to look forward at the trends and tools defining the way we work and live. And how to put them to work.
With an extensive library of content - more than 4000 books and video courses -Packt's mission is to help developers stay relevant in a rapidly changing world. From new web frameworks and programming languages, to cutting edge data analytics, and DevOps, Packt takes software professionals in every field to what's important to them now.
From skills that will help you to develop and future proof your career to immediate solutions to every day tech challenges, Packt is a go-to resource to make you a better, smarter developer.
Packt Udemy courses continue this tradition, bringing you comprehensive yet concise video courses straight from the experts.