AI-300: Azure Machine Learning Operations Engineer Exam Prep

Azure machine learning provides a unified workspace to build, train, evaluate, and deploy classical ml and gen ai models, with a model catalog, data versioning, and real-time or batch deployments.

Deploy an azure machine learning workspace with its underlying resources in a managed resource group, then explore the studio and connections to build and manage models.

Compare compute targets in an Azure Machine Learning workspace, from compute instance and cluster to serverless spark and AKS, and learn when to deploy real-time endpoints or data processing.

Discover how to create and manage compute targets in the Azure ML workspace, including a CPU-based managed compute instance (DS11 v2) for notebooks, with idle shutdown, security, and tagging.

Discover data stores and data assets in Azure ML workspace, using URIs and data assets to version, govern, and reuse data (CSV, PDFs, images) for model training with ML tables.

Compare Azure ML SDK v1 and v2, noting that v2 has higher abstraction, ML table assets, MLflow integration, and YAML pipelines for easier model training and deployment.

Clone the AI-300 GitHub repository into the Azure machine learning notebooks, then open notebooks to perform labs and explore assets, data, and pipelines on a managed compute instance.

Explore data with Spark by building a dataframe from CSV files in Azure ML's serverless Spark, applying a schema, running Spark SQL, and visualizing results with Matplotlib or Seaborn.

Compare Spark DataFrames and Pandas DataFrames to clarify when to use each in the machine learning lifecycle: Pandas for in-memory exploratory data analysis and visualization, Spark for distributed pipelines.

Explore the labs using Jupyter notebooks to run experiments, with PyTorch for deep learning, scikit-learn for models, and Spark, NumPy, Pandas, Matplotlib, and Seaborn for processing and visualization.

Learn linear and shrinkage regression algorithms, including linear, ridge, and lasso, plus logistic regression for binary classification, and how hyperparameters and MLflow enable penalized, accurate models.

Use the Azure Machine Learning Pipeline Designer to build an automobile price prediction model with linear regression, including data prep, a 70/30 split, training, scoring, and evaluation.

Create a real-time inference pipeline from an automobile price prediction model, deploy to an Azure container instance, and expose a real-time web service endpoint to serve predictions via api calls.

Explore real-time inferencing by calling a model behind an Azure container instance with a curl POST to the REST API endpoint, using keys, and viewing predictions and logs.

Explore the anatomy of the machine learning model development lifecycle in Azure Machine Learning workspace through experiments, jobs, and runs, with metrics, parameters, and model registry.

Train a diabetes risk classifier with logistic regression in Azure Machine Learning, using a diabetes data asset, minmax scaling, train/test split, and evaluation via precision, recall, and F1.

Learn to use confusion matrix and ROC curve to evaluate binary classification models. Explore recall, precision, accuracy, and F1 score, and understand area under the curve for stakeholder clarity.

Automate ML workflows with AutoML by feeding data and a target metric. Build a leaderboard, select the top model, and deploy to a real-time endpoint.

Explore automated machine learning in Azure Machine Learning to identify the best diabetes prediction model by training multiple algorithms and deploying the winning ensemble to endpoints.

Discover how MLflow fits into the Azure Machine Learning ecosystem and the MLOps lifecycle, tracking experiments, logging hyperparameters and metrics, registering and deploying models across Azure Machine Learning and Databricks.

Perform hands-on lab tasks with mlflow to track experiments, log iteration and regularization hyperparameters, log accuracy, precision, recall metrics, and register the best diabetes logistic regression model in Azure ML.

Explore how Azure Machine Learning environments define dependencies for training and inference, including curated and custom options built with Docker and conda.yaml, stored in Azure Container Registry.

Create a custom iris environment from a curated mlflow docker image, configure a python 3.10 conda.yaml, and deploy the mlflow model behind a real-time endpoint on Azure Container Registry.

Master hyperparameter tuning with Optuna and MLflow in Azure ML, using a diabetes dataset to optimize a decision tree by tuning depth and bin size for accuracy.

Discover how Azure Machine Learning scripts become reusable components and how to build end-to-end pipelines for feature engineering, training, evaluation, and model logging with yaml workflows.

Create and run a non-parameterised training script for the diabetes dataset using MLflow, logistic regression, and min-max scaling, log metrics, and register the diabetes model in Azure Machine Learning.

Create a parameterized training script by exposing hyperparameters like reg_rate as command-line arguments and injecting them at runtime for reproducible Azure Machine Learning experiments.

Build and execute an Azure machine learning pipeline by preparing data with prep_diabetes.py, training with train_diabetes.py, and logging metrics like accuracy and ROC AUC with MLflow.

Explore the principles of responsible AI—fairness, inclusiveness, reliability and safety, privacy and security, transparency, and accountability—and learn practical guardrails for trustworthy AI systems.

Explore responsible AI metrics like error analysis, cohort and error heat maps, and explanations of feature importance, counterfactuals, and causal analysis for interpretable model evaluation.

Configure a responsible ai dashboard for the diabetes mlflow model, enabling error analysis, counterfactual explanations, causal analysis, and feature importance with the training and testing data.

Explore real-time, batch, and web service endpoints in Azure Machine Learning, highlighting low-latency inference, high-volume batch predictions, and secured deployments behind ACI, AKS, and API gateways.

Master A/B testing and safe rollback in Azure Machine Learning workspace by routing traffic between model deployments behind a single endpoint and using model versioning to roll back when needed.

Practice A/B testing and rollback strategies on a real-time managed endpoint in Azure Machine Learning, routing traffic between diabetes and automobile price prediction models.

Trace the evolution from artificial intelligence to generative AI and large language models, explain transformer-based architectures, foundational models, and building applications on top of them.

Explore how AI agents and compound AI systems extend large language models with external tools and APIs, enabling dynamic, autonomous planning and execution to automate business processes for ROI.

Learn key generative AI jargons, from tokens and input-output cost dynamics to prompts and API terminology. Explore system prompts, user prompts, chat completions API, and multimodal versus unimodal models.

Learn about Azure Machine Learning deployment types for large language models, including global and data zone options, batch and provisioned throughput, and their data residency implications.

Build a simple named entity recognition PromFlow microservice in Azure Machine Learning using prompt engineering and Python scripting to extract entities.

Discover how retrieval augmented generation grounds large language model responses with vector embeddings and an augmented prompt, using Azure AI Search to keep enterprise data up to date.

Explore how Azure AI Search enables retrieval augmented generation and acts as an AI-enabled data store. Indexing Azure Blob Storage data supports AI enrichment with text and image analysis.

Deploy ADA002 text embedding model in the Microsoft Foundry resource to build a multimodal rag index from PDFs in Azure Blob Storage, and enable image verbalization via GPT-4.

Build a rag prompt flow in Azure Machine Learning to connect an AI search index, configure index lookup with Azure AI Search, and generate responses with gpd4o from Microsoft Foundry.

Learn automated, manual, and custom evaluations of a RAC/RAG pipeline in the Azure Machine Learning workspace, using AI assisted groundedness score metrics.

Explore gen-ai evaluation metrics across reference-based, reference-free, and llm-based approaches, including blur, rouge, levenshtein, bert score, groundedness, coherence, relevancy, and rag pipelines.

Run an automated evaluation of a rag PromFlow pipeline to assess groundedness using a downloaded grounding evaluation dataset, GPT-4.0, and a dedicated groundedness metric.

Master prompt engineering techniques, including chain of thoughts prompting, zero short prompting, and few short prompting. Apply best practices like clear syntax, task breakdown, and delimiters to improve LLM outputs.

Optimize retrieval-augmented generation performance for Azure Machine Learning Operations; prepare for the exam with practical strategies and hands-on workflows.

Explore BM25 ranking and semantic L2 re-ranking in Azure AI Search, including how search.score and search.rerankerscore reflect keyword semantics and inverted index context in retrieval augmented generation pipelines.

Explore fixed size with overlap, semantic, recursive, adaptive, and context enriched chunking strategies for preparing documents for a vector search index and retrieval augmented generation pipelines.

Master vector search, keyword search, hybrid search, and semantic re-ranking in Azure AI Search, and understand HNSW vector arrangements and the retrieval pipeline.