XGBoost Deep Dive w/ Python & Pandas | Hands-on Data Science

Develop a supervised regression model to predict Airbnb listing prices by preprocessing data in pandas, performing feature engineering on host and listing metadata, and modeling the price in log form.

Conduct a supervised regression on Airbnb listings using pandas to read csv, handle mixed types, and begin cleaning with host features and property attributes to prepare the target variable.

Define the target variable by cleaning the price column with regex to remove dollar signs and commas, convert to float, and drop zeros; explore log transformation to reduce skew.

Identify raw features from Airbnb data and perform preprocessing for feature engineering, including converting booleans, extracting date features, and preparing categorical and numeric features for price prediction.

Engineer numeric features in Pandas by cleaning percent fields, converting to float, mapping booleans, extracting bathroom counts, and creating a host neighborhood match indicator for machine learning.

The lecture demonstrates feature engineering for categorical variables, including mapping host response time to short labels and counting host verification methods by parsing strings with ast.literal_eval.

This lecture covers feature engineering of date features by computing date differences, converting to datetime with Pandas, and creating numeric date features for machine learning.

Clean up a feature engineering notebook by building a reusable reprocess function that outputs a clean data frame with numeric and categorical features for model training.

Import pre-processing script, read data, and validate the preprocessing steps; transform the target to log, assess categorical cardinality with unique values to anticipate sparse one encoding for those categorical variables.

Apply a train/test split using numpy indexes and the train_test_split function, extract the target variable, and separate numeric and categorical features, handle missing values, and prepare for one-hot encoding.

Apply one-hot encoding to categorical features with sklearn's OneHotEncoder, fit on training data with handle_unknown='ignore', transform train and test data, and combine with numeric features via a sparse stack.

Explore XGBoost parameter tuning and training in Python with pandas, using a matrix input and watch lists, with max depth, learning rate, and lambda tuned via price mean absolute error.

Predict a supervised binary target, user conversion, by transforming 22 million-row raw data into a trainable format, then apply manual ip parameter tuning and automated hyperparameter tuning with Opportune.

Learn to preprocess web analytics data with pandas by turning raw logs into per-user features, filtering training data by date, and preparing a binary conversion target for XGBoost.

Explore feature engineering with pandas by creating pivot-style cross tabs, normalizing visits to percentages, and assembling user-based features for XGBoost modeling.

Learn manual XGBoost hyperparameter tuning in Python, covering train-test split, objective, max depth, learning rate, lambda, subsample, and colsample, with training vs. evaluation monitoring to curb overfitting.

Explore feature importance with XGBoost gain for business insights, and validate performance via cross validation; apply the model to new data with preprocessing and probabilistic predictions.

Evaluate model performance using AUC with a train/test split and XGBoost, comparing predictions to true labels. Compute AUC with scikit-learn and discuss additional metrics for business-related decision making.

Compute precision, recall, and F1-score from a confusion matrix at different probability cutoffs. Assess the base rate and business value when targeting the top 10% most likely to convert.

Learn how to choose a model cutoff from probability predictions using percentiles, compute precision, recall, f1, lift, and base rate, and decide based on business needs.

Automate hyperparameter tuning for XGBoost models with an optimization library, sampling lambda, learning rate, subsample, and max depth to maximize ALC, using cross-validation and early stopping.

Explore supervised multiclass classification by predicting customer credit score in three categories—good, standard, and poor; apply feature engineering and evaluation metrics, including the confusion matrix, for business use.

Explore multiclass classification to predict customer credit scores (good, standard, poor) with xgboost, covering dataset prep, missing values, feature engineering, and date-based train-test splits.

Inspect data to separate numeric and categorical features and apply pandas-based feature engineering for XGBoost. Clean and convert values, addressing strings and missing data.

Engineer categorical features by cleaning occupation, deriving loan-type dummies, and splitting payment behavior into spent and value; assemble numeric and categorical feature lists and save preprocessed data to a pickle.

Train a multi-class xgboost model to predict credit scores from July–August data, performing loading, preprocessing, one-hot encoding, train/test split, model training, evaluation, and feature importance.

Classify StackOverflow questions into over 200 classes using text as features, leveraging title and content, while reducing 3.3 million observations to a smaller dataset for rapid, focused multi-class text classification.

Explore preprocessing for multiclass text classification using stack overflow titles and bodies to predict tags; downsample data with stratified sampling, encode targets, and clean text for effective feature engineering.

Apply stratified train-test splits to text data and engineer features with tfidf and dense embeddings; train models with XGBoost and compare top-5 accuracy across methods.

Explore time series forecasting with XGBoost, fitting daily sales per state using actuals and predicting for groups with hourly MW data, converting raw data into a format and evaluating results.

Learn daily time series forecasting with xgboost, building per-item features, performing a train-test split with a 30-day horizon, and evaluating per-item error with median absolute percentage error.