Machine Learning in Python - Extras

Explore how pipelines connect data extraction, preparation, and modeling to automate the machine learning workflow and produce repeatable, scalable models in Python.

Explore different types of pipelines, including the etl pipeline used by engineers and exploration pipelines for probing data, then review end-to-end and specialized pipelines for project needs.

Explore when to use or skip pipelines and leverage their simplicity, automation, and scalability to reduce redundancy, accelerate iteration, and monitor the workflow as a product.

Learn how to build scikit-learn pipelines by defining transformers and estimators, choosing between pipeline constructor and pipeline object, and applying two stages: data transformation and model fitting.

Examine the shortcomings of the scikit-learn pipeline, including limited compatibility with non-sklearn components and hyperparameter search, and present Neuraxle as an alternative for more flexible pipelines.

Explore building a model with a non-pipeline approach, then learn how pipelines package transformers and estimators to streamline training, evaluation, and scalability.

Build and evaluate a scikit-learn pipeline, including transformers and estimators, to streamline data transformation, scaling, modeling, and visualization in machine learning workflows.

Learn to combine multiple machine learning estimators with a voting classifier to improve accuracy, exploring hard versus soft voting and applying ensembles in pipelines.

Explore building a scikit-learn pipeline that combines multiple transformers and estimators with a voting classifier, improving accuracy and illustrating the workflow.

Learn to visualize scikit-learn pipelines as interactive diagrams, detailing steps, transformers, and estimators, including simple and complex pipelines with voting classifiers.

Explore natural language processing pipelines by chaining data preparation, tokenization, named entity recognition, vectorization, and modeling to build and evaluate classifiers with transformers.

Explore natural language processing pipelines with scikit-learn to build a text classifier model that labels tweets as disaster-related, using both component and pipeline approaches, with preprocessing, vectorization, training, and evaluation.

Learn how to build an NLP text classifier using a pipeline of transformers and a machine learning estimator, including data splitting, vectorization, fitting, predicting, and evaluating accuracy.

Save the pipeline, load it back, and view the entire NLP text classifier pipeline to preserve work and run predictions across updates.

Explore building a multilabel text classifier with a pipeline: transform features, apply a binary relevance model, split data to prevent leakage, and evaluate with classification reports.

Transform a task column into a multilabel format, apply multilabel binarization, and generate binary representation for multi-label classification in Python.

Build multilabel text classification models using movie plot data, vectorization, and tf-idf pipelines. Learn to predict multiple genres per film with binary relevance.

Learn to build PySpark pipelines for natural language processing and machine learning using Spark's in-memory distributed engine, resilient distributed datasets, and a Python wrapper.

Learn to build features from test data with PySpark pipelines, creating transformers and estimators, tokenization, IDF, and vectorized features, then train, evaluate, and predict with a model.

Apply a PySpark pipeline to make predictions by supplying test data, configuring input columns and dates, and using transform to obtain predicted probabilities and outcomes.

Explore imbalanced datasets in machine learning, understand class distribution and causes, and compare under sampling, oversampling, and tree-based models for robust performance in fraud and cancer detection contexts.

Learn to identify and create imbalanced datasets in Python by analyzing class distribution with value counts and visualizations, then apply sampling strategies to simulate and address imbalance for binary classification.

Detect imbalanced datasets by inspecting class distribution and visualizing with Seaborn and Matplotlib; use value counts and percentages to quantify imbalance and guide modeling.

build a baseline model for imbalanced datasets and compare approaches using dummy and logistic regression with cross-validation, then evaluate with precision, recall, F1, and MCC.

Fix imbalanced datasets using oversampling, undersampling, and synthetic samples; switch to tree-based algorithms, and evaluate with accuracy, F1 score, and confusion matrices.

Learn how oversampling raises the minority class using random oversampling, and evaluate its trade-offs within a pipeline using relevant metrics for imbalanced datasets.

Explains under-sampling to balance imbalanced datasets by randomly removing samples from the majority class, discusses trade-offs, and demonstrates applying a random-sample transformer and evaluating a model.

Apply smote to balance imbalanced datasets by generating synthetic minority samples through interpolation between nearest neighbors, then train and evaluate a model to improve accuracy and recall.

Explore fixing imbalanced datasets with Tomek links, a nearest-neighbor technique that removes or moves overlapping samples between classes to balance data and improve model performance.

Combine oversampling and undersampling to address imbalanced datasets, using pipelines and predefined methods to compare performance, and validate with a simple logistic regression model.

Explore modern model evaluation using metrics like precision, recall, F1 score, correlation coefficient, and sensitivity and specificity. Visualize ROC and precision-recall curves to assess true and false positives and negatives.

Explore model interpretation and explanation for machine learning, demystifying black-box predictions with local and global methods, model-specific and model-agnostic tools, and intrinsic versus post hoc approaches.

Learn to interpret machine learning predictions with eli5, analyze feature weights and contributions, and generate clear, component-wise explanations for pipeline models.

This lecture demonstrates using lime, a model-agnostic explainer, to interpret predictions, show feature contributions, and visualize prediction probabilities for test data and images.

Visualize and interpret tree-based machine learning models in Python using dedicated packages, creating decision trees and random forests, evaluating accuracy, and explaining predictions with visual plots and confusion matrices.

Explore model fairness and bias detection in a Python machine learning workflow, using explainable tools and fairness metrics to diagnose bias across demographic groups.

Examine how the decision tree classifier exhibits fairness and bias by evaluating metrics like equal opportunity across minority and female subgroups, highlighting how imbalance affects model fairness.

Explore data drift and concept drift, differentiate data, feature, and label drift, and learn practical detection and monitoring using drift checks and trust reports.

Explore online machine learning for streaming data, contrasting offline batch learning, and build real-time pipelines with river to detect drift, adapt models, and make predictions.

Explore online machine learning with streaming data and incremental learning, building real-time models using a pipeline that handles preprocessing, feature extraction, and classification to distinguish hardware from software.

Discover Turi Create, an apple-developed ML library that simplifies data preparation and modeling. Install with pip, load data from dict or csv, and build classification, clustering, and regression models.

Explore text classification with turicreate by loading data, splitting into train and test sets, training a classifier, and evaluating accuracy and predictions.

Use turi create in python to build a binary classifier for diabetes from a UCI dataset; preprocess features, train a logistic regression model, evaluate accuracy, and predict single samples.

Explore data logging and monitoring for ml projects using mlflow and willcocks, with prometheus, kafka, and splunk to track experiments, logs, and pipeline metrics like counts, distributions, and sample sizes.

Explore visualizing deep learning models built with TensorFlow, including neural network architecture, input features, hidden layers, and activation functions, demonstrated on Iris data with Netron and TensorBoard.

Use Snorkel to label unlabeled data with labeling functions, heuristics, and keywords in a Snorkel Python tutorial, guiding dataset preparation, model training, and evaluation.