Machine Learning with Imbalanced Data

Explore imbalanced datasets, where minority and majority classes create distributions and imbalance degrees, and learn why predicting rare events like fraud, medical diagnoses, equipment malfunction, or oil spills is hard.

Explore how imbalanced classes affect learning by examining class distribution, imbalance ratio, and dataset size. Understand that class separability and within-class sub-clusters drive difficulty more than imbalance alone.

Explore data-level, cost-sensitive, and ensemble approaches to imbalanced datasets, including over-sampling, under-sampling, and SMOTE. Use higher costs for minority misclassifications and ensemble methods to boost minority-class predictions.

Explore performance metrics for imbalanced data, including roc-auc, precision-recall curves, and confusion matrices, and learn how threshold choice and multiclass extensions impact metrics.

Explore accuracy as the common metric for model performance, defined as correct predictions over total, and show why it fails with imbalanced data, especially for minority class, in binary classification.

Compare accuracy of random forest and logistic regression against a baseline majority-predictor using scikit-learn, emphasizing minority-class detection and concluding logistic regression best for positives.

Explore precision, recall, and the F1 score for imbalanced data, focusing on minority class performance, threshold effects, and how to balance metrics for reliable model evaluation.

Compare precision, recall, and F1 scores for imbalanced data using random forest and logistic regression, with scikit-learn and yellowbrick visuals, and optimize thresholds to improve minority-class performance.

Explore the confusion matrix and the false positive discovery rate and false negative discovery rate to understand how true positives, true negatives, false positives, and false negatives shape model thresholds.

Analyze confusion matrices, false positive and false negative rates for a random forest and logistic regression on imbalanced data, and explore thresholds to balance recall and precision.

Explore how balanced accuracy addresses imbalanced datasets by averaging recall across classes. Examine confusion matrix insights, including true positives, and recall to compare class performance beyond overall accuracy.

Estimate the balanced accuracy using scikit-learn on the imbalanced KDD 2004 dataset. Compare it to standard accuracy for the baseline majority-class model, random forest, and logistic regression.

Explore alternative metrics for imbalanced data, including geometric mean, dominance, and the index of imbalanced accuracy, built from recall and true negative rate, with comparisons to precision and ROC-AUC.

Apply geometric mean, dominance, and the index of imbalanced accuracy to compare random forest and logistic regression on imbalanced data, using recall, true negative rate, and balanced accuracy.

Explore the ROC-curve and ROC-AUC to evaluate classifier performance across thresholds, compare models, and understand how threshold choice affects true positive rate, false positive rate, and AUC.

Compare random forest and logistic regression using ROC-AUC score to evaluate discrimination, and plot ROC curves with scikit-learn and Yellowbrick to interpret class-specific performance.

Learn how the precision-recall curve analyzes imbalanced data, showing the precision-recall trade-off at different thresholds, and compare logistic regression and random forest.

Learn to plot precision-recall curves for imbalanced data using scikit-learn and Yellowbrick, compare random forest and logistic regression, and interpret average precision and area under the curve (AUC).

Plot precision-recall curves with scikit-learn and Yellowbrick to compare random forest and logistic regression on the test set, showing logistic regression’s larger area under the curve.

Explore under-sampling methods to balance imbalanced data, comparing fixed and cleaning approaches, balancing ratios, and algorithms like random under-sampling, NearMiss, and instance hardness, and implement in Python with imbalanced-learn.

Describe random under-sampling as a simple, data-agnostic method that balances classes by sampling from the majority until a ratio is reached, using Imbalanced-learn's RandomUnderSampler with a sampling strategy.

Explore how random under-sampling balances imbalanced data with imbalanced-learn, preserves original distributions, and boosts model performance using a random forest on toy and real datasets.

Explore condensed nearest neighbours for boundary-focused under-sampling, using 1-nearest-neighbour training to balance minority and majority classes, with one-vs-one multi-class handling.

Apply condensed nearest neighbours under-sampling to imbalanced data, selecting majority-class samples near the minority class to train a knn and compare performance with a random forest using roc-auc.

Explore Tomek links, pairs of nearest-neighbor observations from opposite classes. Remove the majority observation or the entire TomekLinks pair to reduce boundary noise and improve learning.

Apply Tomek Links from imbalanced-learn to under-sample imbalanced data, removing majority observations or full links, and compare model performance on well-separated and not well-separated datasets.

Apply One Sided Selection from imbalanced-learn to undersample the majority class and compare random forest performance on full versus undersampled data. Observe Tomek Links' effect on data and separation.

Discover edited nearest neighbors, a cleaning method that undersamples the majority class by removing samples near the boundary using a 3-nearest neighbors check.

Implement edited nearest neighbours with imbalanced-learn to undersample data, using 3 neighbors and mode or all criteria, then compare performance and retention of clearly majority class observations.

Explore repeated edited nearest neighbours, a cleaning method using 3-nearest-neighbours to iteratively remove samples whose neighbors disagree, until no more removals or max cycles, via imbalanced-learn fit_resample.

Apply repeated edited nearest neighbours to undersample the majority class using imbalanced-learn, compare with edited nearest neighbours, and observe impact on minority class separation and model performance.

Explore the all k nearest neighbours algorithm, an iterative under-sampling method that removes majority-class samples near class boundaries to balance imbalanced data.

Explore all KNN with imbalanced-learn, comparing it to edited and repeated edited nearest neighbours on toy datasets and a real dataset, and assess undersampling effects on minority classes.

Learn how the neighbourhood cleaning rule under-samples the majority class by removing samples near the boundary with other classes, using edited nearest neighbours and a minority-focused cleaning step.

Apply the neighbourhood cleaning rule with fit_resample to undersample the majority class, using a 3-neighbour rule and a two-to-one threshold.

Explore NearMiss, a fixed under-sampling method from imbalanced-learn that balances majority and minority classes by selecting majority observations based on distance, using three versions.

Explore instance hardness threshold and filtering to remove hard observations, reduce class overlap, and use probability thresholds or percentiles with one-vs-rest for multiclass classification.

Apply Instance Hardness Threshold to a binary classification task. Undersample with the threshold, compare ROC-AUC for random forest and logistic regression, using imbalanced-learn.

Demonstrates instance hardness threshold for multiclass imbalanced data using a one-vs-rest random forest, creating a three-class toy dataset, filtering hard instances by class probabilities, and balancing via auto undersampling.

Assess how undersampling methods affect ROC-AUC for a random forest trained on undersampled data across multiple datasets. Different datasets show varying gains from NearMiss and Instance Hardness.

Explore the trade-offs between fixed and cleaning under-sampling techniques for imbalanced data, and learn when to test multiple methods to improve model performance.

Train the random forest on the under-sampled training set and evaluate on the test set with the original class distribution using cross-validation and the average precision-recall score.

Explore over-sampling methods to address imbalanced datasets by balancing the minority and majority classes. Learn random over-sampling, SMOTE and ADASYN, including SMOTEENC, and how sample generation targets hard-to-classify boundary cases.

Explore random over-sampling to balance imbalanced data by duplicating minority-class observations to a 1:1 ratio, using imbalanced-learn's RandomOverSampler and fit_resample, and adjust balancing strategy for multiclass targets.

Demonstrates how to apply random over-sampling with Imbalanced-learn's RandomOverSampler on a toy dataset created with make_blobs, and visualizes the balance using pandas, matplotlib, and seaborn.

Explore random over-sampling with smoothing to generate new minority-class examples by adding noise guided by class distribution, controlled by a shrinkage factor, and implemented with Imbalanced-learn's RandomOverSampler.

Explore random over-sampling with smoothing using imbalanced-learn's RandomOverSampler to create synthetic minority-class examples and observe dispersion changes as the shrinkage factor varies.

Learn how SMOTE, the synthetic minority over-sampling technique, generates new minority class samples via interpolation between the five nearest neighbors to balance data without duplicating observations.

Apply SMOTE to balance an imbalanced dataset by generating synthetic minority samples with imblearn, demonstrate with make_blobs, show fit_resample effects, and visualize the before/after distributions.

Learn how smote-nc extends smote to handle categorical variables, using knn distances adjusted by the median std deviation, and generate synthetic samples with imbalanced-learn's smotenc.

Demonstrates SMOTE-NC on a toy dataset with numerical and categorical features using imbalanced-learn, oversampling the minority class with fit_resample, preserving class ratios, and generating new synthetic samples.

Learn how SMOTE-N extends SMOTE to work with categorical data using the value difference metric to identify the k nearest minority neighbors, and generate new samples by majority vote.

Explore how to compute distances between categorical variables with the value difference metric, encode with ordinal encoder, and apply SMOTE-N to balance a toy dataset of multiple categorical features.

Explore ADASYN, a minority-class oversampling method that generates synthetic data from harder-to-classify samples with weighting, and contrast it with SMOTE using the full dataset, KNN, and class boundaries.

Demonstrates implementing ADASYN with imbalanced-learn on artificial data in Python, oversampling the minority class with fit_resample, using the sampling strategy auto and neighbours to shape boundary-adjacent synthetic data.

borderline smote refines smote by generating synthetic minority samples near the decision boundary, using knn to select the danger group and offering two variants for interpolation and extrapolation.

Explore Borderline SMOTE with Imbalanced-learn by oversampling a toy dataset using Borderline SMOTE variations 1 and 2, observing synthetic samples near the decision boundary and the new class balance.

Learn how SVM SMOTE uses minority support vectors to interpolate or extrapolate new samples, and compare it with ADASYN and Borderline SMOTE for imbalanced data.

Apply svm smote to oversample the minority class, visualize the support vectors and the decision boundary of a linear svm, and compare the original with the oversampled data.

Explore k-means SMOTE, an intra-class cluster oversampling method that boosts minority regions by clustering data, filtering clusters by imbalance, and generating synthetic samples within clusters to reduce noise.

Demonstrates K-means SMOTE with imbalanced-learn on a synthetic 1,200-sample dataset using make_blobs and KMeans, oversampling minority clusters to rebalance.

Compare over-sampling methods such as random oversampler, SMOTE, ADASYN, borderline SMOTE, and SVM SMOTE on diverse datasets, evaluating roc-auc with a random forest to identify best-performing approaches.

compare over-sampling versus under-sampling for imbalanced data, discuss SMOTE and variants (ADASYN, border-line SMOTE, SVM SMOTE, K-means SMOTE), and emphasize distance metrics and practical trade-offs.

Learn how to correctly set up a classifier using over-sampling or under-sampling to address class imbalance, and evaluate performance on test data with original imbalance, integrating resampling into cross-validation.

Implement oversampling schemes with cross-validation using Imbalanced-learn make_pipeline and scikit-learn cross_validate; compare SMOTE and other oversamplers with a random forest and MinMaxScaler, highlighting varied dataset results.

Combine over-sampling and under-sampling techniques to improve model performance on imbalanced data. Learn how SMOTE, edited nearest neighbours, and Tomek links mitigate noise and preserve information.

Combine over-sampling with under-sampling using SMOTEENN and SMOTETomek in Imbalanced-learn to rebalance datasets and reduce noisy minority observations.

Compare combined over-sampling and under-sampling methods with stand-alone SMOTE variants to assess ROC-AUC performance across imbalanced datasets, with Borderline SMOTE and SVM SMOTE often excelling.

Conclude that there is no universal rule for over-sampling or under-sampling; test original data, random over-sampling, or combinations, while noting nearest-neighbour under-sampling scalability and distance metrics for categorical features.

Explore ensemble methods for imbalanced data by combining bagging and boosting with data level techniques such as under- and over-sampling, including hybrid ensembles and cost-sensitive strategies, with Python implementations.

Discover how ensemble learning improves model performance by combining diverse classifiers that complement each other, using bagging and boosting techniques like random forests and AdaBoost to boost generalization.

Bagging, or bootstrap aggregating, creates datasets by sampling with replacement and trains classifiers on each. It combines predictions by averaging or voting to improve generalization and foster diverse, de-correlated models.

Learn how bagging pairs with under- or over-sampling and SMOTE to balance imbalanced data, train diverse classifiers on balanced subsamples, and ensemble their predictions.

Learn how boosting builds sequential classifiers, weights misclassified observations, and combines predictions with classifier weights (AdaBoost and gradient boosting) to improve performance on imbalanced data.

Explore boosting with data pre-processing to handle imbalanced data, including RUSBoost, SMOTEBoost, RAMOBoost, and ADASYN-inspired RAMOBoost, focusing on under-sampling, synthetic samples, and weighted ensembles.

Explore hybrid methods that combine bagging, boosting, and resampling to boost model performance on imbalanced data, including Balanced Random Forests, EasyEnsemble, and BalanceCascade using AdaBoost.

Compare ensemble methods with and without resampling on imbalanced data, using random forests, boosting, bagging, and easy ensemble in scikit-learn and imbalanced-learn, and evaluate ROC-AUC across datasets.

Wrapping up this section, this lecture compares ensemble methods for imbalanced data, shows boosting's generally strong performance, and highlights scalable, fast resampling like random under-sampling. It stresses metric choice.

Explore cost-sensitive learning and how misclassification costs shape the training objective, using cost matrices to balance imbalanced data and emphasize minority-class accuracy.

Explore misclassification cost, from constant error cost to conditional, time-based, and data and professional costs. Understand how these costs shape classifier decisions in imbalanced data contexts.

Learn to obtain the cost matrix for cost-sensitive learning, using expert input or heuristic estimates, and tune costs as hyperparameters to balance minority and majority classes.

Explore cost-sensitive learning by contrasting direct training methods that embed misclassification costs with meta-learning approaches that adjust data or outputs, including pre-processing and post-processing techniques like meta-cost.

Explore how logistic regression minimises its cost function and incorporates misclassification costs through weighted logistic regression to address imbalanced data.

Explore how to introduce misclassification cost into decision trees by weighting impurity functions (gini, entropy) with class weights. See how these costs extend to random forests for imbalanced data.

Discover cost-sensitive learning in scikit-learn by applying class_weight or sample_weight to misclassification costs. See how balancing techniques and custom class penalties improve ROC-AUC.

Learn to treat the misclassification cost as a hyperparameter and optimize it with grid search on a random forest using the KDD 2004 dataset, balancing ratio, and ROC-AUC evaluation.