Real world data science projects to become data scientist

Explore classification with decision trees and logistic regression on a telco churn dataset using scikit-learn and MLflow, compare models, and understand when trees excel.

Learn how decision trees extend logistic regression for classification, using root splits and leaves to model churn with features like senior citizen and monthly charges.

The lecture explains the measure of impurity in decision trees using the Gini index. It shows how to compute node and split impurity and determine the split by Gini gain.

Compare decision tree types—cart with gini impurity and c5.0 with entropy and information gain—alongside chaid’s chi-squared splits and its limitations with continuous variables.

Walk through code to analyze telco churn with MLflow, encode categorical features, balance data with SMOTE, and compare logistic regression and decision trees using ROC AUC and feature importance.

Compare decision trees with logistic regression using the telco customer churn dataset, explore decision tree fundamentals, Gini index, entropy, and MLflow, then practice on Kaggle datasets to optimize classification models.

Explore ensemble learning techniques—boosting, bagging, Random Forest, AdaBoost, gradient boosting, and XGBoost—evaluate when to use them, and compare models on telco churn data using MLflow for production readiness.

Explore ensemble learning foundations, comparing bagging and boosting with decision trees, random forest, AdaBoost, gradient boost, and extreme gradient boost, using scikit-learn for practical real-world data science.

Learn how random forest trains on random samples, splits data into training and test, builds multiple decision trees with random feature subsets, and uses majority voting to classify.

Explore how AdaBoost builds sequential decision stumps, assigns weights to misclassified samples to guide the next tree, and contrasts this sequential boosting with random forest voting for improved predictions.

Explore gradient boosting and its ada boosting approach, using weak learners and decision trees to iteratively reduce the loss via residuals, with an additive ensemble and learning rate.

Walk through logistic regression, decision trees, and ensemble methods including xgboost on the telco churn dataset, with SMOTE preprocessing and MLflow tracking for roc auc and precision results.

Compare feature importance across decision tree, random forest, AdaBoost, and gradient boost, then use lime to locally explain why a single customer prediction is churn or not churn.

Explore ensemble learning methods, including random forest, AdaBoost, and gradient boosting, and interpret models with LIME, applied to a telco churn dataset, with MLflow tracking and production considerations.

Explore XGBoost on an insurance pricing dataset, perform exploratory data analysis to assess distributions and relationships, examine collinearity, and compare gradient descent with multiple linear regression.

Explore the insurance premium dataset from Kaggle using exploratory data analysis, including histograms, univariate and bivariate analyses, and visualization with Plotly and Seaborn in Python.

Explore how correlation and collinearity reveal relationships between features to guide feature selection for linear models, using Pearson's correlation for numerical data, chi-squared for categorical data, and ANOVA for mixed.

Compute and interpret Pearson correlations for numeric data, visualize relationships with scatter plots and heat maps, and use chi-square and ANOVA to assess independence, collinearity, and expenses.

Learn a base linear regression, extend to multiple linear regression, and evaluate with residuals, normality, and homoscedasticity, using one hot encoding, Yeo-Johnson power transforms, and rmse/r-squared.

Learn gradient boosting with XGBoost, an additive ensemble of weak learners that minimizes loss through gradient descent, using mean squared error for regression and log likelihood for classification.

Compare model performance by compiling CV results and predictions into a data frame, then evaluate training and test data using RMSE and R-squared to show xgboost's improvement over linear models.

Conclude the project by comparing XGBoost with a linear model, review exploratory data analysis, collinearity, correlation, chi-squared, ANOVA, and multiple linear regression, and outline evaluation metrics for future projects.

Analyze Big Mart sales data to predict item outlet sales at the store level using exploratory data analysis, SQL queries, and 8–10 regression algorithms, with data cleaning and visualizations.

Explore end-to-end data science workflow from univariate numerical and categorical box plot analyses to bivariate correlations, feature engineering, encoding, and evaluating multiple regression models.

Compare voting regressor and stacking regressor performance, then explore GAMs and XGBoost, using pipelines, RFE, and hyperparameter search to optimize R2 scores in real-world data science projects.