XGBoost & Random Forest: Decision Trees + Boosting in R

Explore decision trees, random forests, and boosting methods through a teaching case, and learn preprocessing for missing data and how to split training and testing data.

Explore a teaching case on Edutravel that builds predictive models for customer retention from a real dataset to support targeted marketing strategies.

Describe the air travel dataset of the Trayvon case, a 43-variable dataset detailing program types, school attributes, demographics, costs, and the retention outcome.

Set the working directory to the CSV file, import the data into R with a semicolon delimiter, and convert the first column to row names via a data frame operation.

Learn how to change the data type of variables in R, converting numeric categories to factors so decision trees treat them as categories rather than numbers, using as.factor.

Learn how to handle missing data in real-world datasets by filling values and creating surrogate binary indicators, using an R function to fill numeric, factor, character, and date fields.

Learn to preprocess data by combining rare categories in R, using a two-argument function that loops through columns, counts factor levels, and groups levels below a minimum count into others.

Split data into training and testing sets to evaluate predictive models, using a random seed and an 80 percent training and 20 percent testing split with the caret package.

Build your first decision tree with the travel data set using the Citrix function, then compute the confusion matrix to assess accuracy and calculate the area to compare model goodness.

Construct conditional inference trees with the ctree (C3) function, compare to recursive partitioning with rpart, preprocess data, handle missing values, and interpret a practical ctree model in R using Parakeet.

Interpret decision tree results by reading leaf retention patterns to identify high-priority groups and avoid low-retention ones, and compare this approach with logistic regression’s coefficients.

Learn to predict the dependent variable on a testing dataset using the ctree model, estimate retention probabilities, and classify observations based on a threshold to prepare for a confusion matrix.

Construct and interpret a confusion matrix in R by converting predictions to a two-level factor (0 and 1). Assess positives, negatives, false positives, false negatives, and accuracy, sensitivity, and specificity.

Construct and interpret a receiver operating characteristic curve, showing true positive rate and false positive rate as a function of threshold to assess classifier performance.

Learn to estimate and interpret the area under the ROC curve (AUC) for binary classification in R, assess model discrimination, and compare methods across thresholds.

Explore recursive partitioning decision trees using the Arpad function to select the complexity parameter and optimize model structure. Build a confusion matrix to assess accuracy and compare metrics across approaches.

Explore recursive partitioning with rpart to build simple and complex decision trees, compare with conditional approaches using C3 function, and tune complexity parameter cp to balance tree size and fit.

Choose the complexity parameter for ARPAD trees by plotting it against the error and selecting where the error ceases to drop substantially, then prune the tree.

Calculate retention probabilities with the predict function on the testing data, classify observations, and build a confusion matrix to evaluate model accuracy, sensitivity, and specificity, comparing to a C3 model.

Compare ROC curves and AUC for Arpad and C3 models using predicted probabilities and true labels; the C3 model shows higher AUC, indicating better discrimination.

Explore how a random forest improves model accuracy by combining many decision trees through bagging. Learn how to create a random forest, an ensemble algorithm built from multiple trees.

Explore how random forest, a supervised learning algorithm built by bagging multiple decision trees, delivers accurate classification and regression results by using random feature subsets and ensemble averaging.

Build a classification random forest in R with the randomForest package, evaluating importance and proximity, and interpreting variable importance via mean decrease accuracy and Gini impurity.

Compute probabilities of retaining groups with a random forest on the testing data, classify by a 0.607 threshold, and evaluate with a confusion matrix.

Compute and compare ROC curves and AUC for a random forest model in R, plot the ROC, and contrast its performance with a decision tree model.

Explore boosting decision trees with the extra boost package, contrast gradient boosting with random forest, and illustrate construction and parameter tuning through an editable teaching case.

Gradient boosting turns weak learners like decision trees into a strong model through sequential weighted training and compares it with random forests.

Build an XGBoost extreme gradient boosting classifier, prepare data with 60/40 split, convert features to numeric, and tune eta, max_depth, and nrounds before predicting and evaluating with a confusion matrix.

Generate predictions with the predict function and build a confusion matrix for a gradient boosting model. Compare the model's 77% accuracy to the tree and random forest performances.

Celebrate completing the decision trees, random forests, and gradient boosting course in R by applying your foundations to create predictive models with machine learning tools, adding value to your career.

Learn to build predictive models with neural networks for business analytics through a practical credit scoring project using a car dealer dataset in a developing country, with R and code.

Explore the theoretical foundations of neural networks, including origin, architecture, and applications. Describe the three main types—artificial, recurrent, and convolutional networks.

Explore how neural networks learn from data through layered neurons. Identify convolutional, recurrent, and feedforward networks for image recognition, time series forecasting, signal processing in self-driving cars, and anomaly detection.

Artificial neural networks form the basis for deep learning, using feedforward architectures with input, hidden, and output layers to learn non-linear mappings via activation functions.

Explore recurrent neural networks and how looping the hidden state captures sequential information and word dependencies, with parameter sharing across time steps.

This session introduces convolutional neural networks, their automatic filter learning, spatial feature extraction, and parameter sharing, with applications in image and video processing and a teaching case on credit cards.

Explore a credit card switching case that illustrates neural networks in business analytics and credit scoring. Predict defaulters for a car dealer in a developing country using a real database.

Explore a teaching case with Credicars that uses neural networks in business analytics to reveal patterns in loan defaults and how correlations and regression models relate age, income, and education.

Explore credit scoring with neural networks to identify defaulters and make rapid, data-driven credit granting decisions, leveraging historical data for profit and competitive advantage.

Explore how banks use credit scoring to reduce information asymmetry, address adverse selection and moral hazard, and shift from underwriter judgment to statistical models.

Learn how credit scoring uses mathematical techniques to rank customers with a single score, enabling objective decisions and linking risk, response, retention, and revenue.

Explore how credit scoring models use historical data to assess risk, set lending rules, and decide to accept or reject credit offers, while weighing accuracy, speed, reach, privacy, and discrimination.

Describe the credit cards dataset with about 20 variables, including id, contract status, salary day fraction, tenure, and the defaulter flag, and outline a neural network for credit scoring.

Explore a step-by-step credit card switching case in R, including data import and preprocessing. Encode categorical variables, compute correlation, normalize, split data, and compare neural networks with logistic regression.

Neural networks enhance credit scoring by capturing nonlinear patterns and incorporating data such as social media and online behavior, improving accuracy and reducing false positives and negatives.

Learn to import a CSV dataset into R by saving Excel data as CSV, setting working directory, and importing with the proper delimiter to view 28,000 observations of 20 variables.

Identify missing data in the dataset using Excel observations. Convert start date from character to date in R using as.Date with day/month/year format enabling missing data handling for neural network.

Develop a function to fill missing data by type, create surrogate and filled-indicator columns, and apply it to a credit cards data frame to impute dates as 1900-01-01.

Remove unused variables from the data frame by assigning null in the console for neural network prep, specifically contract ID, contract status, and start date.

Learn how to convert categorical variables to numerical in R by inspecting unique values, applying switch-based recoding to region and branch, and casting to numeric for neural networks.

Compute the correlation matrix to identify multicollinearity and use qualitative analysis to decide which variables to keep, with examples like time of employment, down payment fraction, and region or branch.

Learn to normalize data in R for a neural network by applying the range-based formula (x−min)/(max−min), achieving mean near zero and faster convergence.

Set a seed and install the caret package, then split the dataset with createDataPartition for an 80/20 training/testing split to train and evaluate the model.

Explore the neuralnet package in R by configuring the neuralnet function’s key arguments—formula, data, hidden layers, threshold, and error measure—and understand linear output for simple neural networks.

Install and load neuralnet package in R, build a neural network with one hidden layer of two neurons to predict falter flag from training dataset, and inspect the result matrix.

Predict the dependent variable on the testing dataset using the created model, then denormalize predictions and export a real vs predicted defaulter flag CSV for Excel cutoff.

Estimate a cutoff threshold by minimizing total costs from false positives and false negatives using predicted default probabilities and real defaulter data. Identify 0.58 as the optimal cutoff.

Learn to construct a confusion matrix in Excel, calculate true positives, true negatives, false positives, false negatives, and derive sensitivity, specificity, and accuracy for model evaluation.

Run a logistic regression in R using glm with a binomial family to interpret coefficients for default and compare its precision against neural networks, highlighting significant variables and model insights.

Estimate a confusion matrix in R by predicting probabilities from a logistic regression, apply a 0.5 cutoff to classify the defaulter flag as zero or one, and compute accuracy.

Compare confusion matrices for neural network and logistic regression models using a 0.5 threshold to classify defaults, and report accuracy, sensitivity, and specificity.

Explore a neural network solution for a credit default project in R, building a two-hidden-layer net (2,2) and evaluating with a confusion matrix and accuracy.

Complete your neural networks for business analytics foundations, then apply your skills to create predictive models with machine learning tools and add value to your professional career.