Ultimate ML Bootcamp #5: Classification & Regression Trees

Explore CART, the classification and regression tree method, to build simple rules from data and split into homogeneous groups, laying the foundation for random forests.

Explore the CART framework and how decision trees split data into rules, using thresholds and metrics to minimize error, with leaves predicting averages.

Cart uses square footage to predict price via splits, building branches and leaves, and averaging outcomes; it covers SSE, MSE, MAE, RMSE and controls like maxdepth and min samples split.

Examine the theoretical basis of CART, including decision trees, gradient boosting, and cost, entropy, and Gini measures for classification and regression. Analyze how purity and weighted Gini guide branching decisions.

Set up cart, explore from basics to advanced tree methods, and cover hyperparameter optimization, learning curves, visualization, rule extraction, and generating Python, SQL, and Excel outputs.

Model a cart decision tree by selecting a dataset, defining dependent and independent variables, and fitting the model to generate predictions and evaluate accuracy, roc, and auc.

Perform hyperparameter optimization for cart using gridsearchcv, tuning max depth and min sample split to reduce overfitting, and compare best accuracy, F1 score, and ROC AUC scores.

Deploy the final cart model using the decision tree classifier, tune min sample split and max depth via setparams and getparams, and evaluate with cross-validation metrics: accuracy, F1, and AUC.

Rank variables by their contribution to regression or classification tasks, minimize SSA and reduce Gini and entropy, then visualize their importance and note glucose, BMI, and age as top predictors.

Analyze learning curves to assess model complexity in cart decision trees using validation curves, comparing train and test ROC AUC scores to prevent overfitting.

visualize a classification and regression tree, generate a png diagram of the model, and explore splitting criteria such as gini and entropy with adjustable maximum depth.

Extract decision rules from a decision tree by exporting text via the cart funnel model, revealing breakpoints for BMI, glucose, and insulin that predict class zero or one.

Extract python codes, sql queries, and excel formulas to derive decision rules from a decision tree and deploy the model inside the database for in-database predictions.

Explore predictions with Python codes using tree based approaches, extending beyond regression and logistic regression to reveal a new modeling method.

Save and reload machine learning models with joblib, creating and loading pkl files like cart_final.pkl to predict new data, while deploying on servers or the cloud with database integration.