Real-world End to End Machine Learning Ops on Google Cloud

Define MLOps as machine learning plus operations and show how automated pipelines, data versioning, experiment tracking, and model registry connect data science to production.

Understand the end-to-end ML-ops life cycle on Google Cloud, from problem framing and data understanding to EDA, framework selection, training, deployment, CI/CD, feature stores, and monitoring.

Learn how continuous integration and deployment apply to data science and machine learning on Google Cloud, using GitHub or GitLab, Cloud Build, automated tests, and automated deployment with monitoring.

Explain container registry for Docker images and artifact registry for all artifacts, and show how both support CI/CD on Google Cloud by storing built images and artifacts.

Enable container registry, artifact registry, and cloud build APIs in your GCP account to support the labs, then install Python 3.6+ along with Flask and pytest on your local system.

Explore Cloud Run, a serverless, fully managed container service on Google Cloud that scales on demand for web apps, APIs, event-driven workloads, and model serving, with CI/CD for Flask-based apps.

Containerize and deploy a Flask app to Cloud Run, ingest a CSV from a GCS bucket into BigQuery, push container image to container and artifact registries, and enable GitHub CI/CD.

Containerize a Flask app, push the image to container registry and artifact registry, deploy to Cloud Run, and create a BigQuery table to ingest data from a US states CSV.

Demonstrate generating and adapting unit tests for a Flask app with PyTest using ChatGPT, verify 50-row inputs and a 200 response, and run tests locally before deploying to Cloud Run.

Understand cloudbuild as a fully managed, serverless ci cd platform on Google Cloud that builds, tests, and deploys applications, defines yaml-defined steps, and maintains audit logs with GCP integrations.

Learn to deploy a Flask app on Google Cloud with Cloud Build: build the Docker image, push, test, deploy to Cloud Run, and configure permissions.

Create a GitHub repo, upload cloudbuild.yaml, dockerfile, main.py, requirements.txt, and test main, then connect the repo to Cloud Build triggers on the main branch using commit sha for traceability.

Learn end-to-end MLOps on Google Cloud by training an XGBoost model for in-vehicle coupon recommendations, serving predictions through a Flask app, and deploying with Cloud Run and Cloud Build.

Deploy and test a Flask model server locally by loading a pickle artifact, serving predictions at /predict via json payloads, and validating with Pytest across preprocess and port configuration.

Deploy the xgboost coupon model as a Flask service on Cloud Run using Cloud Build. Upload the pickle to a GCS bucket, and update main.py to load from the bucket.

Explore A/B testing for ML models on Cloud Run by comparing model versions, either via separate endpoints or traffic split behind the same endpoint, with a practical 80/20 example.

Deploy a new version of the same machine learning model by tuning xgboost hyperparameters, saving a model pickle, uploading to a gcs bucket, and routing traffic between cloud run revisions.

Productionize a bike rentals regression model using a random forest and deploy a flask api that serves predictions via http post on Google Cloud.

Examine a bank marketing campaign dataset for a binary classification using XGBoost, with preprocessing (label encoding, bucketing, scaling), oversampling, and model training and evaluation, as groundwork for continuous training.

Learn how continuous training automates end-to-end machine learning ops on Google Cloud, validating data, retraining on new data, evaluating performance, deploying updated models, and monitoring pipelines.

Orchestrate end-to-end machine learning pipelines with Airflow or Cloud Composer on GCP to enable continuous training, data validation, retraining, evaluation, and artifact deployment, with metrics in BigQuery and alerts.

Create a GCP composer environment (Airflow 2) and a Vertex AI workbench notebook; enable APIs, set US central 1, and update with requirements.txt via gcloud.

Execute cloud model training on GCP using Jupyter Notebook, log metrics to BigQuery, save artifacts to GCS, and enable continuous training with Airflow.

Execute an airflow dag for a machine learning workflow by validating input data, evaluating metrics against thresholds, and deploying the model artifact to a GCS bucket with BigQuery metrics.

Demonstrates an end-to-end continuous training pipeline in Airflow, from validating CSV inputs to model evaluation, logging results, and storing artifacts in GCS and BigQuery.

We describe a daily airflow continuous training pipeline that deploys new model artifacts to a GCS bucket and serves predictions on Cloud Run. Alerts help diagnose failures and data issues.

Set up automated email alerts for training status below one, signaling missing artifacts in the GCS bucket, by creating log-based metrics with Logs Explorer.

Describe CI/CD for model training: containerize the training script in the dags folder, run tests on cloud build, and deploy to the airflow gcs bucket.

Learn to implement CI/CD for model training on Google Cloud, using cloud-build, pytest, and GitHub, including building Docker images, running tests, and deploying to Airflow DAGs.

Set up cloud build triggers by uploading required files to your GitHub repository, connecting the repo to Cloud Build, and creating a trigger for pushes to the main branch.

Set up end-to-end ML ops on Google Cloud by implementing continuous training for a marketing ROI regression model, with Vertex AI, BigQuery metrics, and a scheduled end-to-end pipeline.

Set up a ci cd pipeline for the data science roi model with cloud build, modularized training code, pytest tests, a GitHub repository, cloudbuild.yaml, and a build trigger.

Deploy a Flask-based model serving app to Google Cloud Run, loading a joblib artifact from local or cloud storage, with logging and curl tests for online predictions.

Explore Vertex AI components and train a data science model end-to-end with Vertex AI training service, containerizing training code and deploying for batch and online predictions via Python SDK.

Explore Vertex AI custom training, containers, and compute options for running jobs on Google Cloud. Compare pre-built versus custom containers, leverage hyperparameter tuning, and save models to Google Cloud Storage.

Examine bike share regression model using UCI bikeshare data, with features like date, year, month, day type, workday or weekend, weather, predicting the count with a random forest regressor.

Containerize the training code, build and push the image, then run a Vertex AI custom training job via web ui or gcloud cli to output a joblib artifact in GCS.

Learn how Vertex AI model registry centralizes versioning, metadata, and deployment to endpoints, with aliases to decouple versions and enable zero-downtime updates in the ML lifecycle.

Submit a custom training job with the Python SDK, upload the joblib artifact to the model registry, and deploy the model to an endpoint for online Vertex AI predictions.

Deploy models from the model registry to a live endpoint, use canary traffic splits to test the challenger, then promote it to production and undeploy the old champion.

Walk through deploying a Cloud Run Flask app to serve online predictions by invoking a Vertex AI endpoint, with batch predictions saved to a GCS bucket and tested via curl.

Set up a batch prediction job on Google Cloud using Cloud Functions triggered by new files in a GCS bucket, invoking Vertex AI batch predict and logging outcomes.

Train a model in Vertex AI, deploy to a model registry, and run batch and online predictions via endpoints, while implementing CI/CD and continuous training on GCP with airflow.

Learn to implement end-to-end ci/cd on Vertex AI using Cloud Build, automating training, model registry upload, and deploying to an endpoint with gcloud commands and cloudbuild.yaml.

Train an in-vehicle coupon recommendation model with xgboost using Vertex AI custom training, then deploy to the model registry for batch predictions and to an endpoint for real-time online predictions.

Train a scikit-learn classifier on Vertex AI to predict product returns, register and deploy the model to an auto-scaling endpoint, and score new orders from GCS before cleaning up.

Explore Kubeflow pipelines as a platform for composing, deploying, and managing end-to-end ML workflows on Kubernetes, with Vertex AI Gist pipelines and seamless integration for data science.

Explore Kubeflow pipelines: two components—component decorator turning functions into steps and the DSL.pipeline decorator defining the pipeline—each run as Docker images and compiled to Json for Vertex AI.

Deploy a simple two-step Kubeflow pipeline for a coupon recommendation model using xgboost, validating the input CSV before training and logging metrics to Google Cloud Storage.

Trigger a Kubeflow pipeline by loading a compiled json and invoking it with the Python SDK to enable continuous training or scheduled runs via cloud scheduler, with deployment options.

Deploy an end-to-end Kubeflow pipeline that validates, trains, and deploys the trained model to a Vertex AI endpoint and model registry.

Deploy a credit scoring classifier using XGBoost in Vertex Pipelines, building a two-task pipeline for validation and training, then an end-to-end pipeline for deployment with metrics logging.

Explore Vertex AI experiments to prototype and compare model performance across different input parameters and hyperparameters, logging metrics and managing runs to identify the best model.

Train an xgboost model with multiple hyperparameter combinations using Vertex AI experiments and Kubeflow pipelines, and compare accuracy, precision, and recall across runs.

Run a lab that compares XGBoost, random forest, and logistic regression classifiers using flexible hyperparameters and Kubeflow pipelines in a Jupyter notebook workflow.

Run two experiments for hyperparameters and model selection in regression task, comparing root mean square error to identify the best model using training_with_hyperparams.py and training_with_models.py in Berkshire regression model folder.

Tune hyperparameters efficiently with Vertex AI, using grid search, random search, or bayesian optimization, to optimize model performance with a defined objective metric and parallel trials.

Execute a hyperparameter tuning job for the bike rentals regression model in Vertex AI, uploading the IPython notebook, preparing a custom trainer with Task.py, and optimizing RMSE with GCS.

Programmatically tune xgboost hyperparameters to maximize accuracy by exploring n_estimators 35–40 and learning_rate 0.2–0.5 for the in-vehicle coupon recommendation model, following the lab assignment.

Explore how Vertex AI explainability reveals why a model predicts outcomes, using integrated gradients, x ray, and Sample Shapley to assess feature contributions for batch and online predictions.

Deploy a trained model to an endpoint with explainability parameters by uploading to the model registry and using sample attribution with a path count to assess feature contributions.

Execute explainability for online predictions by retrieving input attributions and feature contributions, compare baseline and instance outputs, and interpret zero versus nonzero feature impacts.

Enable explainability for batch predictions with the Python SDK by setting generate_explanation and reviewing attributions, baseline values, and feature contributions in the results.

Perform explainability for xgboost models. Train with Vertex AI, upload to model registry with explanation metadata, deploy to a Vertex endpoint, and run online and batch predictions with Shapley attribution.

explore model versioning with the Vertex AI Model Registry, deploying multiple versions under the same parent model for online and batch predictions.

Deploy and manage multiple XGBoost model versions in Google Cloud's model registry, using version aliases, default settings, and parent-child relationships, then route traffic to endpoints and run batch predictions.

Explore Vertex AI feature store, a fully managed centralized repository for features, with entities and entity types, enabling cross-team reuse and data ingestion for credit scoring models.

Create a feature store named fs credit scoring, define four entities and their features in a notebook on Vertex AI workbench, using credit request id as a join key.

Read data from a Vertex AI feature store into a pandas dataframe, using online and batch serving, and specify feature IDs and entity IDs for retrieval.

Explore AutoML on Vertex AI to automate model building, algorithm selection, and hyperparameter tuning, enabling rapid prototyping and production-ready results with pre-trained models on small datasets and minimal coding.

Deploy a tabular AutoML classification model on Vertex AI using a pet dataset, train with an 80/10/10 split, deploy to an endpoint, and generate yes/no probability predictions.

Train and deploy a regression model with AutoML on Vertex AI by uploading CSV data to GCS, training to minimize RMSE, and deploying an endpoint for online predictions.

Explore what generative AI is, its core models—GANs, VAEs, and RNNs—and how they generate realistic content such as images, text, audio, and videos, with latent space that enables controllable generation.

Explore large language models and their capabilities, including translation, summarization, code generation, and chatbots, using Google's Pathways Language Model (Parm) and PM2.

Explore essential keywords and concepts for large language models, including prompts, prompt design, top K sampling, top P sampling, max output tokens, and temperature.

Apply structured extraction by defining pydantic models for invoices and resumes, and use a response schema to return typed data for reliable downstream ingestion.

execute llm experiments with python and Jupyter nb on GCP using Palm API, importing text-bison and chat-bison models to explore prompts and few-shot prompting.

Deploy a flask app using Palm API to Cloud Run for end-to-end text classification, with toxic vs non-toxic and explained prompts, dockerized and tested.

Learn how function calling links Gemini to live systems on Google Cloud using two tools, order status and shipping estimate, through automatic and manual modes in the GenAI SDK.

Learn to generate fashion product descriptions from images by deploying a pre-trained blip image captioning model on Vertex AI and using Palm API to describe each caption.

Implement batch predictions on Vertex AI Workbench using a 200-ticket csv, showing end-to-end batch inference, jsonl inputs, and correlating outputs to ticket ids across five categories.

Implement configurable safety filters and audit blocked responses to safeguard real-user interactions. Reduce costs with context caching and token management, and ensure observability and capacity with logging and evaluation.