Machine Learning Projects A-Z : Kaggle and Real World Pro

Explore a real-world medical cancer dataset from a Google case study, analyze and clean data, and build a machine learning solution to classify cancer types; compare Kaggle and industry perspectives.

Frame a multi-class classification problem from real-world data by prioritizing probability-based predictions, minimizing costly medical errors, ensuring interpretability, and balancing latency with algorithm choices.

This lecture teaches how to preprocess text data for machine learning by cleaning, removing stop words and numbers, normalizing spaces, and imputing missing values before modeling.

Split data into training, cross-validation, and testing sets and use cross-validation to tune hyperparameters, verify class distribution, and prevent leakage before final evaluation.

Explore how log loss serves as the evaluation criterion in multi-class classification, contrast with accuracy, and explain its calculation and interpretation in Kaggle competitions and real-world corporate problems.

Explore how to contrast models using log loss, generate random probabilistic baselines, and evaluate cross-validated and test predictions by selecting the highest probability class.

Evaluate a multi-class model with a nine-by-nine confusion matrix heat map, and interpret precision and recall to understand why accuracy may mislead in multi-class classification.

In this lecture, analyze a categorical gene column and compare one-hot encoding with response encoding, showing how replacing categories with class probabilities reduces dimensionality.

Compare response encoding and one-hot encoding for gene data, showing how response encoding reduces 227 columns to nine class outputs, and implement Laplace smoothing in a practical Python workflow.

Explore how Laplace smoothing and a calibrated classifier improve predictions, using a gene-based logistic regression model, log loss comparison to a random baseline, and handling unseen features with alpha smoothing.

We build a model using only the Jean column with one heart encoding, logistic regression with L2 regularization, and a calibrated classifier to minimize log loss across alpha values.

Explore the second categorical column by analyzing its distribution and applying one-hot and response encoding. Train a logistic regression model, tune alpha, and evaluate log loss to judge its usefulness.

Process the X tax data with word-count features, apply one-hot and response encoding, compute log probabilities and log loss, and set up a calibrated logistic regression model.

Learn data pre-processing before building machine learning models. Implement reusable functions for backlog loss and plotting confusion matrices, and apply response encoding with stacking of columns for integrated datasets.

Explore logistic regression with class balancing and sampling to address imbalanced data and improve log loss. Compare balanced vs unbalanced setups, assess misclassified points, and examine interpretability from model coefficients.

Compare linear and logistic models, switch to hinge loss for interpretability, evaluate with train and cross-validation losses, and analyze classification errors, precision, recall, and confusion matrices, before exploring random forest.

Build a random forest using one-hot encoder and response coding, tuning depth and estimators to minimize log loss and optimize train, test, and cross-validation performance.

Compare random forest models using one-hot encoding and response coding by analyzing training and test results, misclassified points, and the confusion matrix to gauge encoding impact.

Build a stacking model by combining logistic regression, SVM, and neighbors on one hot or response encoded data. Use log loss to select the best Kaggle-ready model.

Apply data cleaning to funding rounds and company datasets by standardizing names to lowercase, handling missing values, and fixing encoding issues to ensure clean, aligned data for analysis.

Load and verify the cleaned data, address missing values, and drop unnecessary columns after an inner join. End with a no-missing data frame, retaining about 77 percent of the data.

Analyze funding types and English speaking countries to identify 5-15 million investments, using mean vs median, box plots, and pivot tables, with sector analysis to follow.

Perform sector analysis by cleaning and merging mapping data, extracting the first category before pipes, normalizing to lowercase, handling missing values, and creating a single sector column.

Explore a loan analysis case study focused on externality data analysis and risk assessment in banking, using historical defaults to guide future loan approvals and minimize losses.

Explore a data case study: understand data, clean missing values, drop columns with missing data, analyze loan factors, and convert features like loan amount and interest rate to numeric.

Explore data analysis steps for loan risk, including variable reduction from a data dictionary, 0/1 encoding, and trend analysis of default rates by grade, loan purpose, and loan amount.

Explore segmented analysis of loan risk factors by purpose, debt concentration, and employment level, using plots and loop calculations to compare loan amount, interest rate, DTI, and income.

Learn a simple linear regression approach to model car prices in the American market, using company name as the independent variable and prepared datasets from UCLA and Kaggle.

The lecture guides exploring a car pricing dataset, using a data dictionary and correlation heatmaps and pair plots, then cleaning, preparing features, and building and evaluating a model.

Clean and process data by converting to categorical, extracting car brands from names, and fixing spelling with regular expressions. Create dummy variables, drop originals, scale data, and split 70/30.

Build and evaluate linear regression models with train/test splits, compare r-squared and adjusted r-squared across feature counts, and use hyperparameter tuning with plotting to select a simpler, robust model.

Evaluate the final six-feature linear regression model, examine the near-zero error distribution, assess predictor correlations with a heat map, and explore using rfe to refine price prediction.

Work through a real world Kaggle case study on text comparison, exploring question and answer platforms, duplicates detection, upvotes, and accessing datasets with log loss evaluation.

Explore data preprocessing for a question similarity task: import libraries, load the dataset, inspect shape and columns, identify missing values and duplicates, and understand question IDs for duplicates.

Learn to frame duplicate question detection as binary classification, optimize with log loss, and use confusion matrices and high-probability thresholds to balance accuracy, latency, and user trust.

Analyze the data distribution by counting zero, one, and unique values. Compute duplication and non-duplication percentages and identify the most repeated questions.

Analyze data distribution by comparing unique and duplicate questions, identify missing values, replace them with spaces, and introduce feature engineering with derived columns to prepare for modeling.

Derive new text features like frequency, word counts, verbs, and common words, then compare basic and advanced feature engineering to identify meaningful columns.

Learn practical text preprocessing for machine learning: clean web page text, remove tags and punctuation, apply stemming and stop-word removal, and normalize contractions and numbers to improve similarity detection.

Learn advanced feature engineering after preprocessing, including tokenization, stop words, and metrics like common word count (cwc) and ctc to refine text features.

Compare question 1 and question 2 using word counts, first and last word checks, and absolute differences, then apply fuzzy wuzzy to measure text similarity between questions.

Learn how edit distance measures character insertions, deletions, and shifts between strings, and how partial and token-based ratios address partial matches and substrings in real-world cases.

Discover methods for measuring document similarity by focusing on common words, intersections, and metrics like the longest substring ratio and fuzzy wuzzy.

This lecture demonstrates converting 15 advanced and basic features into a programmatic feature set, then analyzes their distribution and visualizes them in 2D and 3D to assess duplicate question detection.

Apply idf-weighted vectors using spaCy and glove within a linear program, to build a 15-feature data frame for duplicate detection.

Build and evaluate machine learning models by securely loading data with SQLite, converting categorical features to numeric, performing train-test splits, and comparing log loss for logistic regression, SVM, and boosting.

Compare logistic regression with L2 regularization and hyperparameter tuning to boosting and linear SVM for high-dimensional data; evaluate log loss on train and test to monitor bias and overfitting.

Explore building a multi-label prediction model for Stack Overflow questions using Kaggle data, with focus on title, body, and tags, and evaluating with precision and recall.

This lecture clarifies mean f1 score for multi-label classification, comparing micro and macro averaging, and shows computing micro precision and recall by aggregating tp, fp, and fn across all classes.

Explore hamming loss as mismatches divided by total samples with a prediction versus actual example. Store data in sqlite, remove duplicates, and prepare a clean dataset for analysis.

Analyze the frequency of unique tags, visualize top trends with a histogram and a word cloud, and apply preprocessing—lowercasing and code-snippet separation—for NLP modeling.

Explore transforming multi-label problems into binary relevance (one-vs-rest), classifier chains, or label powerset, and learn data representations with scikit-learn to handle sparse, small datasets.

Selects a subset of features from a high dimensional dataset using frequency-based scoring, achieving up to 99% coverage with about 5,500 top features while discussing memory trade-offs and data sparsity.

Examine high-dimensional modeling, balancing imputation and feature reduction against compute limits, while comparing logistic regression to random forest and increasing title weighting for better performance.

Explore a logistic regression model with L1 regularization on a smaller dataset of 500k records and 500 features, and compare performance via F1 scores, regularizer tweaks, and gram lengths.