AI/ML Foundations for Absolute Beginners (AgenticAI + MLOps)

Traditional ML Models

A Beginner's Guide

MLOps, LLMOps & Agentic AI Bootcamp

School of DevOps

Once Upon a Time in ML Land...

Imagine a kingdom with different types of helpers:

• Linear Regression: The straight-line drawer

• Decision Trees: The question asker

• Random Forests: The village council

• Neural Networks: The brain mimicker

Each helper has special talents for solving different problems!

Linear Regression: Drawing the Line

Real-Life Story: Sarah wants to predict house prices in her neighborhood.

She notices:

• Bigger houses cost more (usually)

• Older houses cost less (usually)

• More bathrooms mean higher prices (usually)

Linear regression draws the "best-fit line" through all these relationships!

Linear Regression: How It Works

The House Price Example:

House Price = $100,000 (starting point) + $100 × Square Feet + $15,000 × Number of Bedrooms - $2,000 × House Age (years)

For a 2,000 sq ft, 3-bedroom, 10-year-old house: Price ≈ $100,000 + $200,000 + $45,000 - $20,000 = $325,000

When to Use Linear Regression

Perfect for:

• Predicting numbers (prices, temperatures, sales)

• Understanding which factors matter most

• Simple, explainable predictions

Not great for:

• Complex patterns or relationships

• Yes/no questions

Real-world uses:

• Predicting sales based on advertising spend

• Estimating how temperature affects ice cream sales

• Forecasting crop yields based on rainfall

Logistic Regression: Yes or No Questions

Real-Life Story: A doctor named Luis needs to predict if patients have a certain disease.

He looks at:

• Age

• Blood pressure

• Family history

• Certain symptoms

Logistic regression helps him calculate the probability of disease!

Logistic Regression: The S-Curve

Unlike a straight line, logistic regression creates an S-shaped curve:

• Output is always between 0 and 1 (a probability)

• Below 0.5: Probably "No"

• Above 0.5: Probably "Yes"

Email Example:

• 0.95 = 95% chance this is spam

• 0.03 = 3% chance this is spam (probably not spam)

When to Use Logistic Regression

Perfect for:

• Yes/No predictions

• Spam or not spam

• Approved or denied

• Fraud or legitimate

Real-world uses:

• Credit card approval

• Email spam filtering

• Disease diagnosis

• Customer churn prediction (will they quit your service?)

Decision Trees: Playing 20 Questions

Real-Life Story: Carlos works at a bank approving loans. He creates a simple flowchart:

Is income > $50,000? ├── Yes → Is debt-to-income < 30%? │ ├── Yes → APPROVE │ └── No → Check credit score └── No → Is credit score excellent? ├── Yes → APPROVE └── No → DENY

This is exactly how a decision tree works!

Decision Trees: Simple But Powerful

Why people love decision trees:

• Easy to understand (even for non-technical people)

• Can show the tree to others

• Works with all types of data

• Makes decisions similar to humans

Like following a recipe: If this, then do that. Otherwise, do something else.

When to Use Decision Trees

Perfect for:

• When you need to explain your model

• Mixed types of data

• When rules are important

Real-world uses:

• Customer segmentation

• Diagnosis systems

• Risk assessment

• Determining eligibility

Downside: They can become too specific to training data (overfitting)

Random Forests: Asking the Crowd

Real-Life Story: Instead of asking one doctor about your symptoms, imagine asking 100 doctors and taking a vote on their diagnosis.

That's a random forest!

How it works:

1. Create many decision trees (the "forest")

2. Each tree sees slightly different data

3. Get predictions from all trees

4. Take the majority vote (or average)

Random Forests: Wisdom of the Trees

Why it works:

• Individual trees might make mistakes

• But they tend to make different mistakes

• The majority is usually right

• Less likely to overfit than a single tree

Like a democracy of trees!

When to Use Random Forests

Perfect for:

• When you need high accuracy

• When a single decision tree overfits

• When you have a reasonable amount of data

• When you want feature importance rankings

Real-world uses:

• Credit risk assessment

• Predicting disease outcomes

• Recommendation systems

• Fraud detection

Support Vector Machines: Finding Boundaries

Real-Life Story: Maria needs to separate ripe vs. unripe fruits on a conveyor belt.

SVM finds the best dividing line by maximizing the "gap" between the groups.

Like: Finding the fairest way to split a room between two roommates, leaving maximum buffer space between their areas.

When to Use Support Vector Machines

Perfect for:

• Clear separation between categories

• Working with limited data

• High-dimensional data

• When you need a clear boundary

Real-world uses:

• Image classification

• Text categorization

• Handwriting recognition

• Detecting manufacturing defects

Neural Networks: Brain-Inspired Learning

Real-Life Story: Alex wants to recognize handwritten digits (0-9) automatically.

Simple rules don't work because everyone writes differently!

A neural network can learn the patterns by seeing thousands of examples, similar to how you learned to recognize digits as a child.

Neural Networks: Simplified

The Restaurant Analogy:

• Input Layer: Taking customer orders

• Hidden Layer: Kitchen staff processing orders

• Output Layer: Final dishes delivered to customers

Data flows through the network, being transformed at each step!

When to Use Neural Networks

Perfect for:

• Complex patterns (images, speech, text)

• When other models struggle

• When you have lots of data

• When accuracy matters more than explainability

Real-world uses:

• Image recognition

• Voice assistants

• Translation

• Recommendation systems

Choosing the Right Model: The Vehicle Analogy

Bicycle (Linear/Logistic Regression):

• Simple, easy to understand

• Gets you there on flat, smooth roads

• Limited capability but practical

Car (Decision Trees/Random Forests):

• More versatile

• Handles more conditions

• Good balance of power and practicality

Airplane (Neural Networks):

• Powerful, handles complex journeys

• Requires more resources

• Best for difficult terrain

Model Evaluation: Did It Work?

The Dating Analogy:

Accuracy: How often did you pick a good match?

• 90% accuracy = 9 out of 10 dates were good matches

Precision: When you thought there was a match, how often were you right?

• High precision = When you say "we'll click," you're usually right

Recall: How many good matches did you find out of all possible matches?

• High recall = You find most of the people you'd be compatible with

Regression Metrics Made Simple

Predicting House Prices:

Mean Absolute Error (MAE):

• On average, how many dollars are you off by?

• MAE = $15,000 means predictions are off by $15,000 on average

R² (R-Squared):

• How much better are you than just guessing the average price?

• R² = 0: No better than guessing the average

• R² = 1: Perfect predictions

• R² = 0.7: 70% better than guessing the average

Summary: ML Models Simplified

Remember:

• Linear Regression: Drawing the best straight line through data

• Logistic Regression: Yes/no probability predictions

• Decision Trees: Flowchart of yes/no questions

• Random Forests: Committee of decision trees voting

• Neural Networks: Brain-inspired pattern recognition

The right model depends on your specific problem

Large Language Models (LLMs)

A Complete Beginner's Guide

MLOps, LLMOps & Agentic AI Bootcamp

School of DevOps

What Are Large Language Models?

LLMs are AI systems that:

• Read and write text like humans

• Complete your sentences

• Answer your questions

• Write stories, emails, and code

• Translate languages

• Summarize long documents

Popular examples: ChatGPT, Google Gemini, Claude, Llama

The Library Analogy

Imagine an LLM as a massive library:

Traditional ML Models:

• A small bookshop with specific books

• Good at one topic (like only sports books)

Large Language Models:

• A giant library containing millions of books

• Has "read" almost everything on the internet

• Can talk about nearly any topic

• Combines knowledge in new ways

A Day with LLMs: Real-Life Uses

Meet Priya, a marketing manager in Bengaluru:

Morning: Asks an LLM to summarize 5 market research reports

Noon: Uses it to draft email responses to clients

Afternoon: Creates social media post ideas for a new campaign

Evening: Translates content to Hindi and Tamil

"It's like having a versatile assistant who can handle almost any text-based task!"

How Do LLMs Work? The Prediction Game

LLMs play a sophisticated "guess the next word" game:

Example:

• Input: "The capital of India is..."

• LLM predicts: "New Delhi"

Through billions of such predictions, LLMs learn:

• Grammar and language rules

• Facts about the world

• How to reason and respond

• Cultural references

What Are "Tokens"?

Tokens are the bite-sized pieces of text an LLM processes:

• Parts of words or whole words

• Punctuation marks

• Special characters

Example: "I love eating samosas" might become: ["I", "love", "eat", "ing", "samo", "sas"]

Think of tokens as:

• The individual pieces LLMs read and write

• Like letters in a Scrabble game

The Token Limit: Short-Term Memory

Context Window = How much text an LLM can "see" at once

The Conversation Analogy:

• Like a person who can only remember the last few minutes of conversation

• Earlier parts get forgotten when new information comes in

• Newer models remember more (larger context windows)

ChatGPT: Can remember about 8,000 words Claude, GPT-4: Can remember 50,000+ words

What Are "Parameters"?

Parameters are the "knowledge knobs" in an LLM:

• Adjustable values that store what the model has learned

• More parameters = more storage for language patterns

• Modern LLMs have billions of parameters

The Brain Cell Analogy:

• Like connections between brain cells

• Each connection stores a tiny piece of knowledge

• Billions of connections create a thinking system

The Memory Palace Analogy for Parameters

Imagine parameters as rooms in a memory palace:

• Small model (1 million parameters): A house with 1,000 rooms for storing knowledge

• Medium model (7 billion parameters): A city with 7 million buildings

• Large model (175 billion parameters): A country with 175 million buildings

Each "room" stores a tiny piece of language knowledge

Why "Large" Matters in LLMs

1. Data: Trained on trillions of words from:

• Books, articles, websites

• Code repositories

• Wikipedia, news, forums

2. Parameters: Billions of adjustable values

• GPT-3: 175 billion parameters

• Like having 175 billion "memory cells"

3. Computing: Thousands of specialized chips

• Training can cost millions of dollars

• Like building a supercomputer just for language

The History of LLMs: A Short Story

Chapter 1 (Before 2017): Simple models that predicted words, often making mistakes

Chapter 2 (2017): The "Transformer" invention changes everything

Chapter 3 (2018-2020): Models grow larger, showing surprising abilities

Chapter 4 (2022-2023): ChatGPT makes LLMs mainstream

Chapter 5 (2023-Present): Models become multi-talented (text, images, code)

The Transformer: What Makes Modern LLMs Possible

Before Transformers:

• Models processed text one word at a time

• Like reading a book with a tiny flashlight, seeing one word at a time

With Transformers:

• Models look at all words at once and understand relationships between them

• Like reading a book with the full page visible, seeing how words connect

This was the breakthrough that enabled modern LLMs!

The Transformer Architecture: A School Classroom

Imagine a classroom:

Self-Attention: Students listening to each other discuss a topic, focusing on important points

Multi-Head Attention: Different study groups focusing on different aspects (grammar, content, context)

Feed-Forward Networks: Students individually processing what they heard

Result: Better understanding of the whole text

Attention: The Magic of "Looking at Relationships"

The Party Conversation Analogy:

When you hear "bank" in a conversation, you need context to understand the meaning:

"I deposited money at the bank" → Financial institution "I went fishing by the river bank" → Edge of a river

Self-Attention allows the model to:

• Look at all words together

• Understand how they relate to each other

• Focus on relevant words for context

• Disambiguate meanings based on context

The Attention Mechanism: Simple Example

Sentence: "The man who wore a mask couldn't be recognized"

Without Attention: Model might connect "mask" and "recognized" but miss their relationship

With Attention:

• When processing "recognized"

• Pays strong attention to "mask" and "couldn't"

• Understands the mask prevented recognition

Like connecting the dots between related words, even if they're far apart

Pre-training: The General Education Phase

The School Analogy:

Pre-training = General Education (K-12)

• Learning broadly about many subjects

• Building a foundation of knowledge

• No specific career goal yet

• Takes many years (or enormous computing power for LLMs)

The Process:

• Model reads trillions of words from the internet

• Predicts missing or next words

• Adjusts its parameters to improve predictions

• Eventually learns language patterns and knowledge

Fine-tuning: The Specialized Training Phase

The School Analogy (continued):

Fine-tuning = Specialized Education (College/Professional Training)

• Focused on specific skills

• Builds on general knowledge

• Prepares for specific career

• Takes less time than general education

Example:

• Start with pre-trained model that understands language

• Show it examples of customer service conversations

• It learns the specific style and knowledge for customer support

• Result: A specialized customer service AI

Pre-training vs. Fine-tuning: A Restaurant Story

Meet Rahul, who wants to become a chef:

Pre-training (General Cooking Knowledge):

• Spends years learning all cuisines

• Masters basic techniques

• Understands ingredients

• Learns food science

• Very expensive and time-consuming

Fine-tuning (Specialization):

• Takes additional training in South Indian cuisine

• Much shorter training period

• Uses existing cooking knowledge

• Less expensive

• Results in a specialized South Indian chef

Foundation Models vs. Task-specific Models

Foundation Models:

• General-purpose, like a skilled worker with basic training

• Can do many tasks reasonably well

• Examples: GPT-4, Llama, Claude

Task-Specific Models:

• Specialized, like a professional in a specific field

• Excel at one particular task

• Examples: Code-generation models, medical assistants

Like the difference between:

• A general handyman who can fix many things

• A specialized electrician who's expert in one area

Talking to LLMs: Introduction to Prompts

A "prompt" is simply what you say to an LLM to get a response

The Tour Guide Analogy:

• LLM is like a tour guide in a new city

• Without specific directions, they might show you random sights

• Clear directions get you exactly where you want to go

Example:

• Vague: "Tell me about India"

• Specific: "Write a 2-paragraph summary of India's space program achievements"

Prompt Engineering: The Art of Clear Instructions

Bad Prompt: "Write something about climate"

Good Prompt: "Write a 300-word explanation of climate change impacts in Mumbai for a 10th-grade student. Include 3 specific examples and potential solutions."

What Makes It Better:

• Clear length (300 words)

• Specific topic (climate change in Mumbai)

• Defined audience (10th-grade)

• Exact requirements (3 examples, solutions)

Prompt Engineering: Using Examples

Teaching by Example:

Showing examples helps LLMs understand exactly what you want:

Convert these statements to Hindi: English: Hello, how are you? Hindi: नमस्ते, आप कैसे हैं? English: Where is the railway station? Hindi:

Like training a new employee by showing them examples of good work

The Role You Assign: Setting the Stage

You can tell an LLM to assume a specific role:

You are an experienced math teacher explaining concepts to 8-year-old children. Explain multiplication in simple terms.

Other useful roles:

• "You are a cybersecurity expert..."

• "You are a chef specializing in North Indian cuisine..."

• "You are a helpful coding assistant..."

This helps frame how the LLM responds

Controlling LLM Output: Temperature

"Temperature" controls how creative or predictable the LLM is:

Low Temperature (0.1-0.3):

• More predictable, focused responses

• Good for factual answers, code, specific instructions

• Like following a recipe exactly

High Temperature (0.7-1.0):

• More creative, varied, surprising responses

• Good for brainstorming, creative writing

• Like experimenting in the kitchen

Temperature: A Story of Two Chefs

Chef Anil (Low Temperature = 0.2):

• Always follows recipes precisely

• Consistent results every time

• Excellent for standard dishes

• Not very creative or experimental

Chef Prisha (High Temperature = 0.8):

• Uses recipes as inspiration

• Results vary and surprise

• Creates unique combinations

• Sometimes makes unusual choices

Both are valuable for different situations!

What Are "Top-p" and "Top-k"?

These control which words the LLM considers when generating text

The Restaurant Menu Analogy:

Top-k = limiting choices to k most likely options

• Like only considering the 10 most popular dishes on a menu

Top-p (nucleus sampling) = considering options until reaching a probability threshold

• Like considering dishes that make up 80% of all orders

Both help control randomness and quality of generated text

LLM Limitations: Hallucinations

"Hallucinations" = confidently stating incorrect information

The Overconfident Student Analogy:

• A student who makes up answers rather than saying "I don't know"

• Sounds convincing but may be completely wrong

Example: Question: "Who was the first female astronaut from India?"

Hallucinated Answer: "Ritu Karidhal was the first female astronaut from India, completing her space mission in 1997."

Reality: Kalpana Chawla was the first Indian-born woman in space (2003). Ritu Karidhal is a rocket scientist, not an astronaut.

Why Do Hallucinations Happen?

LLMs are trained to:

1. Continue text in plausible ways

2. Sound confident and fluent

3. Provide complete-looking answers

But they don't actually:

• Truly understand facts

• Know when they don't know

• Check their knowledge

It's like a student who prioritizes having an answer over having the correct answer

LLM Limitations: Knowledge Cutoff

LLMs only know information up to their training cutoff date

The Frozen Library Analogy:

• Like a library that stopped receiving new books after a specific date

• Very knowledgeable about things before that date

• Completely unaware of events after that date

Example: "ChatGPT's knowledge cutoff is January 2022, so it doesn't know about events that happened after that date unless recently updated."

LLM Limitations: Context Window

Context window = how much text an LLM can consider at once

The Short-Term Memory Analogy:

• Like a person who can only remember the last few minutes of conversation

• Can only "see" a limited amount of text at once

• If content exceeds this limit, early information is forgotten

Analogy: Trying to read a book through a small window that only shows a few pages at a time

More Limitations of LLMs

Reasoning Limitations:

• Struggle with complex logic puzzles

• May make simple math errors

• Can miss contradictions in their own text

Bias Issues:

• Reflect biases in their training data

• May generate stereotyped content

• Can treat topics unevenly

Lack of True Understanding:

• Don't truly "understand" meaning

• Pattern matching rather than comprehension

• No genuine knowledge or beliefs

Solving the Knowledge Problem: RAG

Retrieval-Augmented Generation (RAG):

• Combines LLMs with information retrieval

• Searches databases or documents for relevant information

• Includes this information in the prompt

• Results in more factual, up-to-date answers

The Assistant with a Reference Library:

• LLM alone = Smart person making educated guesses

• LLM with RAG = Smart person with access to reference library

RAG: How It Works in Simple Terms

1. User asks a question: "What were the key announcements in the 2024 Indian budget?"

2. System searches a database of reliable sources (finds recent articles about the 2024 budget)

3. System includes this information in the prompt: "Based on the following information: [budget details]... Answer the question about the 2024 Indian budget"

4. LLM generates response using this reliable information

Result: More accurate, up-to-date answers!

LLMs in Production: Size Matters

The challenge: LLMs are HUGE

GPT-3 (175B parameters):

• ~350GB model size

• Expensive to run

• Requires specialized hardware

Solutions:

• Cloud APIs (use someone else's infrastructure)

• Model compression (make models smaller)

• Specialized hardware (optimize for AI)

Running LLMs: Your Options

1. Using Cloud APIs:

• OpenAI, Google, Anthropic, etc.

• Pay per use (per token)

• No technical hassle

• Limited control

2. Running Your Own:

• Open-source models (Llama, Mistral)

• Full control

• Higher technical complexity

• Significant hardware needs

• Privacy advantages

Making LLMs Smaller and Faster

Quantization:

• Reducing numerical precision

• Like compressing a high-res photo to medium quality

• 2-4x smaller with minimal quality loss

Distillation:

• Creating smaller "student" models

• Like teaching a summary of knowledge to a new model

• Faster but slightly less capable

Pruning:

• Removing less important connections

• Like editing down a long essay to key points

LLMOps: Running LLMs in Production

Unique Challenges:

• Managing prompts like code (versioning)

• Monitoring for harmful outputs

• Detecting hallucinations

• Keeping costs under control

• Handling high traffic efficiently

The Factory Analogy:

• Traditional MLOps = Regular factory

• LLMOps = Factory with special requirements for fragile, expensive materials

Real World Example: LLM-Powered Customer Service

Deepika implements an LLM system for her e-commerce company:

The System:

• Uses fine-tuned LLM for customer service

• Connects to product database (RAG approach)

• Has guardrails for sensitive topics

• Falls back to human agents when uncertain

Benefits:

• 24/7 support coverage

• Handles 70% of queries automatically

• Reduces wait times

• Frees human agents for complex cases

Getting Started with LLMs: First Steps

1. Start with cloud APIs:

• OpenAI, Claude, etc.

• Low technical barrier

2. Experiment with prompts:

• Learn prompt engineering basics

• Try different instructions and formats

3. For developers:

• Try open-source models like Llama

• Explore model hosting options

• Learn about fine-tuning and RAG

Summary: Large Language Models (LLMs)

Key Takeaways:

• LLMs are AI systems that understand and generate human-like text

• They work by predicting tokens (pieces of text) based on patterns

• Transformer architecture with attention revolutionized language models

• Parameters are the "knowledge storage units" in LLMs

• Pre-training teaches general language, fine-tuning adds specialization

• Prompts are how we instruct LLMs

• Limitations include hallucinations, knowledge cutoff, and context windows

• RAG enhances LLMs with external knowledge sources

Agentic AI Basics

Understanding AI That Takes Action

MLOps, LLMOps & Agentic AI Bootcamp

School of DevOps

What is Agentic AI?

Agentic AI systems:

• Can take actions on their own to complete tasks

• Make decisions based on their goals

• Use tools and interact with the world

• Remember past actions and learn from them

• Work independently with minimal human supervision

Think of them as: AI assistants that don't just answer questions but can actually do things for you!

The Personal Assistant Analogy

Traditional AI (like simple chatbots):

• Like an advisor who can only give information

• "The best restaurant nearby is Spice Garden."

LLMs (like ChatGPT):

• Like an advisor who can have conversations and create content

• "Here's a detailed restaurant recommendation and directions."

Agentic AI:

• Like a personal assistant who can actually make the reservation for you

• "I've booked a table at Spice Garden for 7:30 PM and added it to your calendar."

The Four Key Components of Agentic AI

1. Goals: What the agent is trying to achieve

2. Tools: Capabilities the agent can use

3. Memory: Information the agent can store and recall

4. Planning: How the agent decides what to do next

The Road Trip Analogy:

• Goals: Your destination

• Tools: Your car, GPS, and credit card

• Memory: Remembering routes and past experiences

• Planning: Mapping the journey and making adjustments

Real-Life Example: Meet AIRA

AIRA (AI Research Assistant):

Deepak, a researcher in Mumbai, uses AIRA to help with his work:

1. Deepak asks AIRA to research recent advances in renewable energy

2. AIRA searches the web, finds relevant papers, and summarizes them

3. AIRA creates a bibliography in the proper format

4. AIRA monitors for new publications on this topic

5. When new research appears, AIRA notifies Deepak

All of this happens with minimal oversight from Deepak!

How Agentic AI Differs from Regular AI

Traditional ML Models:

• Make specific predictions

• Run once when called

• No memory between uses

• Limited to one task

LLMs (like plain ChatGPT):

• Generate text responses

• Limited to conversation

• Can't take real-world actions

Agentic AI:

• Makes decisions and takes actions

• Persistent with ongoing tasks

• Remembers past interactions

• Uses multiple tools to achieve goals

Goals: The Driving Force

Goals give agents purpose and direction.

Types of Goals:

• Task-based: Complete a specific task (book a flight)

• Optimization: Maximize or minimize something (find the cheapest flight)

• Maintenance: Keep something in a desired state (monitor price changes)

• Learning: Gather information (research travel destinations)

Like humans, agents need clear goals to be effective!

The Importance of Clear Goals

Unclear Goal: "Find me something about travel"

Clear Goal: "Find me the 3 cheapest flights from Delhi to Bangkok departing next weekend, compare their amenities, and book the one with the best balance of price and comfort."

The GPS Analogy:

• Without a specific address, GPS can't give you directions

• Similarly, agents need specific goals to work effectively

• The clearer the goal, the better the result

Tools: How Agents Interact with the World

Tools are the abilities that allow agents to take actions.

Common Tools:

• Web browsers and search engines

• APIs (weather, maps, flight booking)

• Data analysis tools

• Calendar and email access

• Document creation and editing

• Code execution environments

Just as humans use tools to extend their capabilities, agents use tools to accomplish tasks.

Real-World Tool Examples

Weather API: Get current weather or forecast

get_weather(location="Mumbai", forecast_days=5)

Search Tool: Find information online

web_search(query="best restaurants in Bengaluru")

Calendar Tool: Schedule appointments

calendar_add_event(title="Team Meeting", date="2025-03-10", time="14:00")

Each tool has specific inputs and outputs the agent must understand.

Memory: Remembering What Matters

Without memory, agents would start from scratch every time.

Types of Memory:

• Short-term: Current conversation or task information

• Long-term: Knowledge from past interactions

• Episodic: Specific events or actions taken

• Procedural: How to perform certain tasks

The Human Memory Analogy: Just as you remember your friends' preferences when buying gifts, agents remember your preferences and past interactions.

Memory in Action: A Story

Meet Priya and her AI Shopping Assistant:

First Interaction: Priya: "I'm looking for gifts for my mother." Agent: "What does your mother like?" Priya: "She loves gardening and the color purple." Agent: [Stores this information in memory]

One Month Later: Priya: "I need a birthday present for my mother." Agent: "Based on our previous conversation, would you like to see purple gardening tools or botanical gifts?"

This continuity makes agents feel more helpful and personalized.

Planning: The Decision-Making Process

Planning is how agents decide which actions to take to achieve their goals.

The Chess Player Analogy:

• Consider possible moves (actions)

• Evaluate outcomes of each move

• Choose the best path forward

• Adjust strategy as the game evolves

Types of Planning:

• Sequential: Step-by-step approach

• Hierarchical: Breaking goals into sub-goals

• Reactive: Responding to changing conditions

• Probabilistic: Dealing with uncertainty

Planning Methods in Agentic AI

Common Approaches:

1. Chain-of-Thought: The agent thinks through steps logically

To book a flight, I first need to search available options, then compare prices, then check baggage policies...

1. Tree of Thoughts: The agent considers multiple possible paths

If I search by price, I might find cheaper options. If I search by duration, I might find more convenient options. Let me try both approaches and compare...

1. ReAct (Reasoning + Acting): The agent alternates between reasoning and taking actions

The Agent Loop: How Agents Work

The fundamental process of agentic AI:

1. Observe: Gather information about the current state

2. Think: Process information and decide what to do

3. Act: Take action using available tools

4. Learn: Update knowledge based on results

5. Repeat: Continue this loop until the goal is achieved

This loop allows agents to work through complex tasks step by step.

The Agent Loop: Real Example

Goal: Book a restaurant for dinner tonight

1. Observe: User wants restaurant for 4 people, Indian cuisine, in Bengaluru 2. Think: Need to find restaurants, check availability, make reservation 3. Act: Search for "top-rated Indian restaurants in Bengaluru" 4. Observe: Found 5 matching restaurants 5. Think: Need to check availability for each 6. Act: Check reservation system for each restaurant 7. Observe: 3 restaurants have availability at 8:00 PM 8. Think: Should present options to user 9. Act: Present the 3 options with ratings and menus ...and so on until the reservation is completed

Agentic AI Architecture: The Building Blocks

Main Components:

• LLM Brain: Core reasoning capability (often GPT-4 or similar)

• Tool Library: Collection of available actions

• Memory System: Storage for past interactions and knowledge

• Planning Module: Decision-making process

• Execution Engine: Carries out the planned actions

• Safety Guardrails: Ensures responsible behavior

LLM as the Brain of Agents

The Central Role of LLMs:

• LLMs provide the reasoning capability

• They understand natural language instructions

• They can generate thoughts and action plans

• They interpret results of actions

• They communicate with users

Think of an LLM as the "brain" that powers the agent's thinking, while tools are the "hands" that allow it to act.

Types of Agentic AI Systems

Based on autonomy level:

• Semi-autonomous: Require confirmation for actions

  • "I found these flights. Should I book the 9:00 AM departure?"

• Fully autonomous: Complete tasks independently

  • "I've booked your flight based on your preferences. Confirmation sent to your email."

Based on task scope:

• Specialists: Excel at specific domains (travel booking, coding)

• Generalists: Handle a wide range of tasks with less depth

Popular Agentic AI Examples

AutoGPT:

• One of the first popular autonomous agents

• Can run multiple steps without intervention

• Uses GPT models for reasoning

BabyAGI:

• Simple, task-focused agent framework

• Breaks down goals into manageable tasks

• Demonstrates basic autonomous behavior

Microsoft Copilot:

• Commercial agentic AI for productivity

• Helps with writing, coding, and data analysis

• Integrates with Microsoft tools and services

The Emergence of Multi-Agent Systems

Multi-agent systems have multiple AI agents working together:

• Different agents with specialized roles

• Agents communicate and collaborate

• Can solve more complex problems

• Inspired by human team collaboration

The Restaurant Kitchen Analogy:

• Head chef (coordinator)

• Line cooks (specialists)

• Waitstaff (interface with customers)

• Dishwashers (maintenance)

Together they accomplish what would be difficult for a single agent.

Real-World Multi-Agent Example: Software Development

Development Team of AI Agents:

• Product Manager Agent: Defines requirements and priorities

• Architect Agent: Creates high-level design

• Developer Agents: Write specific code modules

• Testing Agent: Finds bugs and issues

• Documentation Agent: Creates user guides

Each specialized agent handles part of the process while communicating with others.

Agentic AI in Business: Real Applications

Customer Service:

• Agents that handle the entire customer journey

• Can look up information, make changes to accounts, process refunds

Research & Analysis:

• Agents that gather information from multiple sources

• Analyze data and create comprehensive reports

Administrative Support:

• Scheduling meetings based on availability

• Managing email and organizing information

• Preparing documents and presentations

Software Development:

• Writing, testing, and debugging code

• Creating documentation

• Deploying applications

Agentic AI: Challenges and Limitations

Current Challenges:

1. Tool Brittleness:

   • Tools break or change over time

   • APIs may have unexpected behaviors

2. Planning Limitations:

   • Difficulty with complex, long-term planning

   • Sometimes gets stuck in loops

3. Safety Concerns:

   • Potential for harmful actions if not properly constrained

   • Security risks with powerful tools

4. Alignment Problems:

   • Ensuring agent goals match human intentions

   • Avoiding unwanted side effects

The Future of Agentic AI

Coming developments:

• More capable reasoning: Better planning and problem-solving

• Extended memory: Richer, more useful long-term memory

• Better tool use: More reliable and diverse tool integration

• Improved collaboration: Both human-AI and AI-AI teamwork

• Specialized domains: Industry-specific agents with deep expertise

The long-term vision: AI systems that can handle increasingly complex real-world tasks with minimal human supervision.

Building Your First Agent: Starting Simple

Begin with a focused agent:

1. Choose a specific domain

   • E.g., "Research assistant" or "Data analyzer"

2. Define clear tools

   • Search tools, data processing, scheduling

3. Start with supervision

   • Confirm actions before execution

4. Add more abilities gradually

   • Expand tools and autonomy as you gain confidence

Remember: Even simple agents can provide significant value!

Agentic AI and MLOps: The Connection

Unique MLOps challenges for Agentic AI:

• Tool management: Versioning and maintaining tool connections

• Monitoring agent behavior: Tracking actions and decisions

• Agent testing: Evaluating complex, multi-step behaviors

• Safety guardrails: Implementing and updating constraints

• Multi-agent orchestration: Managing teams of agents

MLOps for agents is more complex than for traditional ML or even LLMs!

Case Study: Agentic AI in Healthcare

Dr. Mehta at a hospital in Delhi uses an agentic healthcare assistant:

Capabilities:

• Monitors patient vital signs from hospital systems

• Alerts doctors to concerning changes

• Retrieves relevant patient history

• Suggests possible diagnoses based on symptoms

• Orders routine tests based on protocols

• Schedules follow-up appointments

Result: Dr. Mehta can focus on complex cases while the agent handles routine monitoring and administrative tasks.

Ethical Considerations in Agentic AI

Key questions to consider:

• Transparency: Do users know they're interacting with an agent?

• Control: Can people easily override agent actions?

• Privacy: How is data handled during agent operations?

• Bias: Are agent recommendations fair across different groups?

• Accountability: Who is responsible if an agent makes mistakes?

Responsibility: Creating agents requires thinking through these ethical implications carefully.

Getting Started with Agentic AI: Tools & Frameworks

Popular frameworks:

• LangChain: Tools for building agent workflows

• AutoGPT: Open-source autonomous agent framework

• CrewAI: For building multi-agent systems

• Microsoft Semantic Kernel: Framework for AI agents

Cloud Platforms:

• OpenAI's Assistants API

• Google's Agents for Vertex AI

• Anthropic's Claude with tools

These provide the building blocks for creating your own agents.

Summary: Agentic AI Basics

Key takeaways:

• Agentic AI systems can take actions to complete tasks autonomously

• Four key components: Goals, Tools, Memory, and Planning

• Agent Loop: Observe, Think, Act, Learn, Repeat

• LLMs provide the reasoning "brain" of agents

• Tools connect agents to external systems and capabilities

• Memory allows for continuity across interactions

• Multi-agent systems combine specialized agents for complex tasks

• MLOps for agents requires special considerations

Thank You!

Questions?

Next Module: Core MLOps, LLMOps & AgenticOps Principles

School of DevOps

What is MLOps: The Origin Story

Slide 1: Title Slide

Title: What is MLOps: The Origin Story Subtitle: How AI Success Demanded Operational Excellence School of DevOps™

[Visual suggestion: A dawn breaking over a technological landscape, representing the emergence of a new discipline]

Slide 2: The AI Revolution Begins

Title: The AI Revolution Begins

[Visual: Timeline showing explosion of ML/AI adoption from 2012-present]

The Gold Rush Era:

• 2012: Deep learning breakthrough in ImageNet competition

• 2015-2017: AI investment grows 300%

• 2018-2020: Organizations rushing to implement AI

• 2021-Present: AI becomes business-critical

The Promise: ML/AI would transform businesses through:

• Automated decision making

• Predictive capabilities

• Personalized customer experiences

• Operational optimization

Slide 3: Great Expectations vs Reality

Title: Great Expectations vs. Reality

[Visual: Split screen showing expectations (rocket ship) vs. reality (rocket on launch pad with technical issues)]

The Dream:

• Train a model

• Deploy it

• Watch the magic happen

• Profit!

The Reality:

• 87% of ML projects never reach production

• Average 9+ months from model to deployment

• 50%+ of models deployed fail to deliver expected value

• Technical debt accumulates rapidly

The ML Project Lifecycle was broken.

Slide 4: The Shocking ML Project Statistics

Title: The Cold, Hard Truth

[Visual: Dramatic statistics visualization with stark contrasts]

ML Projects by the Numbers:

• $15.7 Trillion: Projected global AI economic impact by 2030

• 83%: Data scientists frustrated by model deployment challenges

• 90%: Organizations struggling with AI implementation

• 55%: ML projects that devolve into "technical debt monsters"

• 70%: Reduction in time-to-value when proper operational practices are in place

Source: Various industry studies 2019-2023

Slide 5: The 3AM Crisis

Title: The 3AM Crisis

[Visual: Data scientist being awakened by emergency alert on phone]

It's 3AM. Sarah, the lead data scientist, gets a frantic call:

"The recommendation engine is suggesting winter coats to users in Australia... in summer!"

The Painful Questions:

• Which version of the model is running?

• What data was it trained on?

• How did it pass testing?

• Why wasn't the drift detected?

• How quickly can we roll back?

Without systematic operational practices, there were no good answers.

Slide 6: The ML Production Gap

Title: The Production Gap

[Visual: Chasm between "ML Development" cliff and "Production" cliff with failed bridges]

Data Science World:

• Jupyter notebooks

• Experimentation focus

• Local development

• Static datasets

• Academic metrics

Production World:

• Scalable infrastructure

• Reliability requirements

• Resource constraints

• Dynamic data

• Business metrics

The Gap: Most organizations lacked the bridge between these worlds.

Slide 7: The Hidden Complexity

Title: The Hidden Complexity of ML Systems

[Visual: Iceberg diagram showing visible ML model at top but massive operational requirements below water]

Visible Part (10%):

• Model architecture

• Training code

• Initial accuracy metrics

Hidden Complexity (90%):

• Data collection pipelines

• Feature engineering

• Data validation

• Experiment tracking

• Model versioning

• Deployment infrastructure

• Monitoring systems

• Governance frameworks

• Retraining mechanisms

Slide 8: The Birth of MLOps

Title: MLOps: A New Discipline Emerges

[Visual: The convergence of three streams - ML, DevOps, and Data Engineering]

MLOps emerged at the intersection of:

Machine Learning: Science of creating predictive models

DevOps: Practices for reliable software delivery

Data Engineering: Systems for data processing and management

Result: A new discipline focused on the reliable delivery of ML value in production.

Slide 9: What is MLOps?

Title: What is MLOps?

[Visual: Definition with key words highlighted]

MLOps Definition: "MLOps is a set of practices at the intersection of Machine Learning, DevOps, and Data Engineering aimed at deploying and maintaining ML systems in production reliably and efficiently."

Key Aspects:

• Bridges development and operations

• Standardizes the ML lifecycle

• Automates repetitive processes

• Enables reproducibility

• Ensures governance

• Maintains quality

Slide 10: The Restaurant Analogy

Title: If ML Were a Restaurant...

[Visual: Split screen restaurant showing chaotic kitchen vs. organized operation]

Without MLOps:

• Chefs (Data Scientists) creating dishes with no standardized recipes

• No inventory system for ingredients (data)

• Kitchen staff (Engineers) struggling to recreate dishes

• Customers (Users) getting inconsistent meals

• Health inspectors (Compliance) unable to trace sources

• No way to scale successful dishes

With MLOps:

• Recipe versioning and standardization

• Ingredient tracking and quality control

• Consistent preparation processes

• Scalable kitchen operations

• Transparent food safety

• Continuous improvement

Slide 11: The 3 Pillars of MLOps

Title: The 3 Pillars of MLOps

[Visual: Three pillars supporting a platform labeled "Reliable ML in Production"]

Pillar 1: Continuous Integration/Continuous Delivery (CI/CD)

• Automated testing, building, and deployment

• Integration of ML assets with applications

• Consistent deployment processes

Pillar 2: Orchestration & Automation

• End-to-end workflow management

• Coordination of pipeline components

• Dependency handling

Pillar 3: Monitoring & Management

• Model performance tracking

• Data drift detection

• System health monitoring

• Automated interventions

Slide 12: MLOps Core Practices

Title: MLOps Core Practices

[Visual: Circular workflow diagram showing interconnected practices]

Version Everything:

• Code, data, models, configurations

Automate Pipelines:

• Training, testing, deployment

Track Experiments:

• Parameters, metrics, artifacts

Monitor Continuously:

• Performance, drift, resource usage

Enable Governance:

• Lineage, documentation, compliance

Standardize Environments:

• Development, testing, production

Slide 13: The Technical Debt Monster

Title: The Technical Debt Monster

[Visual: Monster made of tangled code, data, and model fragments growing larger]

Google Research Warning: "Machine learning systems have a special capacity for incurring technical debt because they have all the maintenance problems of traditional code plus an additional set of ML-specific issues."

ML-Specific Debt:

• Data dependencies

• Configuration complexity

• Experimentation without tracking

• Undocumented feature engineering

• Manual deployment processes

• Lack of monitoring

• Entangled systems

MLOps is the debt prevention strategy.

Slide 14: ML Lifecycle vs Software Development

Title: ML Lifecycle vs. Software Development

[Visual: Side-by-side comparison of two workflows with key differences highlighted]

Traditional Software:

1. Requirements

2. Design

3. Implementation

4. Testing

5. Deployment

6. Maintenance

ML Development:

1. Problem framing

2. Data collection & preparation

3. Feature engineering

4. Model selection & training

5. Evaluation

6. Deployment

7. Monitoring & retraining

Key Differences:

• Data dependency

• Non-deterministic behavior

• Continuous retraining needs

• Dual validation (code AND model)

Slide 15: Business Value of MLOps

Title: The Bottom Line: Business Value

[Visual: Graph showing performance improvements with MLOps implementation]

Quantifiable Benefits:

• Faster: 60-70% reduction in time-to-deployment

• Better: 40% improvement in model performance

• Reliable: 65% fewer production incidents

• Scalable: 3-4x more models in production

• Compliant: 90% reduction in governance issues

Strategic Benefits:

• Competitive advantage through faster innovation

• Higher ROI on ML investments

• Reduced operational risk

• Greater trust in AI systems

Slide 16: The MLOps Maturity Journey

Title: The MLOps Maturity Journey

[Visual: Five-step progression path from manual to fully automated]

Level 0: Manual Process

• Manual data preparation, training, deployment

• No versioning or reproducibility

Level 1: ML Pipeline Automation

• Automated training pipeline

• Basic versioning

• Manual deployment

Level 2: CI/CD Pipeline Automation

• Automated testing and deployment

• Experimentation tracking

• Basic monitoring

Level 3: Automated Operations

• Automated drift detection

• On-demand retraining

• Comprehensive monitoring

Level 4: Full Automation

• Auto-triggered retraining

• Self-healing systems

• Automated governance

Slide 17: Early Adopters' Stories

Title: The Pioneers' Advantage

[Visual: Logos and brief success metrics from early adopter companies]

Netflix: Created Metaflow, reducing model deployment time by 60%

Uber: Built Michelangelo, enabling 10,000+ daily model predictions

Facebook: Developed FBLearner, supporting 1 million+ model runs daily

Airbnb: Implemented Bighead, increasing experiment velocity by 4x

Google: Created TFX, reducing model incident rate by 70%

Common Thread: Systematic operational practices were the key to scale.

Slide 18: The Changing Landscape

Title: The Evolving AI Landscape

[Visual: ML/AI landscape evolution showing growing complexity]

2015-2018: Traditional ML Focus

• Custom models

• Structured data

• Single models

• Centralized development

2018-2021: Deep Learning Expansion

• Neural networks

• Unstructured data

• Model ensembles

• Distributed training

2021-Present: Foundation Models & Agents

• Large language models

• Multimodal systems

• Agentic capabilities

• Tool-using AI

Each phase brought new operational challenges.

Slide 19: Looking Ahead

Title: The Evolution Continues

[Visual: Road extending toward horizon with signposts for MLOps, LLMOps, and Agentic AIOps]

Next in Our Journey: Understanding how MLOps evolved to address new AI paradigms:

MLOps → LLMOps → Agentic AIOps

As AI systems evolved from traditional ML to foundation models to autonomous agents, operational practices needed to evolve as well.

In our next deck: We'll explore this evolution and the unique operational challenges of each paradigm.

Slide 20: Questions for Reflection

Title: Reflect on Your Organization

[Visual: Reflective scene with thought bubbles containing key questions]

1. Where does your organization sit on the MLOps maturity journey?

2. What is your biggest pain point in moving ML from development to production?

3. How much time could your team save with proper MLOps practices?

4. What would be the business impact of deploying models twice as fast?

5. Which MLOps practice would create the most immediate value for your team?

The Evolution: From ML to LLMOps to Agentic AI

A Journey Through Time

School of DevOps

The Dawn of Machine Learning

Where It All Began

• In 1950, Alan Turing proposed the "Turing Test" to determine if a machine could exhibit intelligent behavior

• The term "Machine Learning" was coined by Arthur Samuel in 1959 at IBM

• Early algorithms were simple but revolutionary - like teaching a computer to play checkers

Much like how we teach children through examples rather than explicit programming

The Foundation Years (1960s-1990s)

Building Blocks of Intelligence

• 1960s: Perceptron model introduced - the first neural network architecture

• 1970s: "AI Winter" begins as expectations outpace results

• 1980s: Expert Systems gain popularity in specific domains

• 1990s: Support Vector Machines and Decision Trees emerge

This was like the construction phase of the Delhi Metro - laying the groundwork despite skepticism

The Renaissance Period (2000s-2010)

From Theory to Practice

• 2006: Deep Learning revolution begins with Geoffrey Hinton's breakthrough

• Computational power increases (CPUs → GPUs)

• Big data becomes accessible

• ML transitions from academic curiosity to practical applications

Similar to how smartphones evolved from luxury items to everyday necessities in India

The Industrialization Era (2010-2015)

From Lab to Production

• Companies begin implementing ML at scale

• Key challenge emerges: How do we deploy models reliably?

• Data scientists create models, but production deployment is difficult

• The "model in a laptop" problem becomes apparent

Like the gap between creating a blueprint for a flyover in Bangalore and actually constructing it in real-world conditions

The Birth of MLOps (2015-2018)

Bridging the Gap

• Term "MLOps" emerges as a discipline

• Combines DevOps principles with ML workflows

• Addresses key challenges:

  • Reproducibility

  • Versioning

  • Deployment

  • Monitoring

  • Governance

Just as IT companies in India adopted Agile methodologies, ML teams needed their own operational framework

MLOps in Action

A New Way of Working

• Version Control: Not just code, but data and models too

• CI/CD for ML: Automated testing and deployment pipelines

• Monitoring: Detecting data drift and model performance degradation

• Governance: Tracking lineage and ensuring compliance

Similar to how IRCTC transformed from a manual ticketing system to a robust digital platform

The Rise of Transformer Models (2017-2019)

A Paradigm Shift

• 2017: Google introduces the "Attention Is All You Need" paper

• Transformer architecture becomes the foundation for modern NLP

• 2018: BERT demonstrates unprecedented language understanding

• 2019: GPT-2 showcases impressive text generation capabilities

This was the equivalent of moving from regular trains to the Vande Bharat Express - a quantum leap in capability

The LLM Revolution (2020-2022)

Language Models Take Center Stage

• 2020: GPT-3 (175B parameters) demonstrates emergent abilities

• Foundation models become accessible via APIs

• Applications explode across industries

• LLMs show versatility beyond traditional ML models

Like how UPI transformed digital payments in India, LLMs began transforming how we interact with technology

New Operational Challenges

Beyond Traditional MLOps

• Prompt engineering becomes critical

• Model evaluation becomes more qualitative

• Need for Retrieval-Augmented Generation (RAG)

• Hallucination detection and prevention

• Unique monitoring requirements

This was like moving from running a roadside dhaba to managing a 5-star hotel - entirely new levels of complexity

LLMOps Emerges (2022-2023)

Evolution of the Operational Framework

• LLMOps extends MLOps principles for language models

• Focus shifts to:

  • Prompt versioning and management

  • Vector databases for knowledge retrieval

  • Fine-tuning workflows

  • Evaluation frameworks for language tasks

  • Responsible AI controls

Similar to how Swiggy/Zomato had to develop new operational frameworks beyond traditional restaurant management

LLMOps vs. Traditional MLOps

Key Differences

Traditional MLOps LLMOps Data-centric Prompt-centric Structured metrics Qualitative evaluation Model training from scratch Fine-tuning & adapters Linear pipelines Complex workflows with retrieval Technical monitoring Ethical monitoring

The Dawn of Autonomous Agents (2023-Present)

From Models to Actors

• LLMs gain the ability to use tools and APIs

• Models can plan sequences of actions

• Memory and reasoning capabilities emerge

• Systems can autonomously complete complex tasks

Like the evolution from manual rickshaws to self-driving cars - not just following instructions but making decisions

Agentic AI Emerges

The Next Frontier

• Autonomous systems that can:

  • Plan multi-step tasks

  • Make decisions with reasoning

  • Use tools and APIs

  • Maintain memory and context

  • Learn from feedback

Similar to how a skilled project manager in an IT company coordinates multiple teams to deliver a complex project

Operational Needs for Agentic Systems

Beyond LLMOps

• Tool orchestration and management

• Multi-agent coordination systems

• Memory and state management

• Safety and alignment guardrails

• Human-in-the-loop feedback mechanisms

This is like moving from managing a single cricket match to orchestrating the entire IPL season

The Operational Spectrum

A Unified View

• MLOps: Managing traditional ML systems

• LLMOps: Managing language model systems

• AgenticAIOps: Managing autonomous agent systems

Each builds upon and extends the previous framework while introducing new capabilities and challenges.

The Future Landscape

What Lies Ahead

• Increasingly autonomous systems

• Multi-modal agents (text, vision, audio)

• Human-AI collaboration frameworks

• Specialized operational platforms

• Standardization of AgenticAIOps practices

Like how we've moved from basic feature phones to smartphones to now anticipating ambient computing environments

Key Takeaways

The Evolution Continues

1. Machine learning has evolved from academic theory to transformative technology

2. Each evolutionary stage (ML → LLM → Agents) brings new operational challenges

3. MLOps → LLMOps → AgenticAIOps represents the parallel evolution of operational frameworks

4. The complexity and autonomy of AI systems continues to increase

5. Mastering these operational frameworks is essential for successful AI implementation

Thank You!

School of DevOps

Let's begin our journey into mastering MLOps, LLMOps, and AgenticAIOps

"The best way to predict the future is to create it."

DECK 2: COMPARING APPROACHES (20 slides)

Slide 1: Introduction

Title: A Tale of Three Approaches

[Visual: Three distinct paths of MLOps, LLMOps, and Agentic AI]

In this section, we'll explore the key differences between:

• Traditional MLOps

• LLMOps

• Agentic AI Operations

Understanding these distinctions will help you navigate which approach is right for your projects.

Slide 2: Terminology Check

Title: Speaking the Language

[Visual: Dictionary or glossary concept with key terms]

MLOps: Operational practices for traditional ML models focused on structured data and supervised learning.

LLMOps: Operational practices specifically for Large Language Models with unique considerations for prompts, retrieval, and output quality.

Agentic AI: Operational practices for autonomous systems that can execute multi-step tasks using tools, planning, and memory.

Operational Excellence: The practices, processes, and tools that enable AI systems to reliably deliver business value in production.

Slide 3: The Evolution Timeline

Title: The Evolution: ML → LLM → Agentic AI

[Visual: Timeline with key milestones and shifts]

Traditional ML (2000s-2010s):

• Focus: Classification, regression, clustering

• Data: Structured, tabular

• Key capability: Prediction on specific tasks

LLMs (2018-Present):

• Focus: Text understanding and generation

• Data: Vast text corpora, multimodal

• Key capability: General language understanding

Agentic AI (2023-Future):

• Focus: Task execution and problem-solving

• Data: Tool interactions and feedback

• Key capability: Autonomous completion of goals

Slide 4: Problem Types & Applications

Title: Different Problems, Different Solutions

[Visual: Application categories mapped to technologies]

MLOps Excels At:

• Fraud detection

• Demand forecasting

• Recommendation systems

• Image classification

• Time series analysis

LLMOps Excels At:

• Content generation

• Summarization

• Translation

• Question answering

• Conversational interfaces

Agentic AI Excels At:

• Research tasks

• Complex workflows

• Tool-based operations

• Multi-step problem solving

• Autonomous execution

Slide 5: Core Components - Conceptual View

Title: The Building Blocks

[Visual: High-level conceptual components of each approach]

MLOps Core Components:

• Data management

• Model development

• CI/CD pipelines

• Model registry

• Deployment automation

• Performance monitoring

LLMOps Core Components:

• Foundation model management

• Prompt engineering

• Knowledge retrieval

• Response evaluation

• Safety & governance

• Cost optimization

Agentic AI Core Components:

• Task planning & reasoning

• Tool integration

• Memory systems

• Feedback loops

• Safety guardrails

• Orchestration

Slide 6: Data Handling Differences

Title: It All Starts With Data

[Visual: Different data processing approaches]

MLOps Data Focus:

• Structured/tabular data

• Feature engineering & selection

• Data quality validation

• Training/testing splits

LLMOps Data Focus:

• Text corpora and knowledge bases

• Vector embeddings

• Retrieval strategies

• Context management

Agentic AI Data Focus:

• Tool-specific data

• Memory storage

• Interaction history

• Multi-modal information

Slide 7: Primary Challenges

Title: Every Hero Has Their Nemesis

[Visual: Challenge icons for each approach]

MLOps Challenges:

• Data drift & quality

• Model reproducibility

• Deployment complexity

• Monitoring at scale

LLMOps Challenges:

• Hallucinations

• Context limitations

• Prompt consistency

• Cost management

Agentic AI Challenges:

• Task planning reliability

• Tool integration complexity

• Safety constraints

• Reasoning failures

Slide 8: Key Differences - Quick View

Title: At a Glance: Key Differences

[Visual: Comparison table with icons]

AspectTraditional MLOpsLLMOpsAgentic AIPrimary FocusCustom model optimizationPrompt & retrievalTask executionCore InputStructured dataText & promptsGoals & tasksMain OutputPredictionsText responsesCompleted tasksKey MetricAccuracyResponse qualityTask success rateMain Cost DriverTrainingInferenceTool operations

Slide 9: Organizational Impact

Title: How They Change Your Organization

[Visual: Organizational chart showing different team structures]

MLOps Impact:

• Bridges Data Science and Engineering

• Requires DevOps skillsets

• Centers on model lifecycle

LLMOps Impact:

• Creates need for prompt engineers

• Shifts focus to content quality

• Emphasizes knowledge management

Agentic AI Impact:

• Demands tool integration expertise

• Introduces autonomous system oversight

• Requires cross-functional collaboration

Slide 10: When to Use Each

Title: Choosing Your Path

[Visual: Decision flowchart for approach selection]

Choose MLOps When:

• Working with structured data

• Need high precision predictions

• Have specific, well-defined problems

• Require full model customization

• Have abundant labeled data

Choose LLMOps When:

• Working with text, images, or speech

• Need language understanding

• Have content generation requirements

• Want to leverage foundation models

• Need flexible, general solutions

Choose Agentic AI When:

• Need autonomous task execution

• Have complex multi-step workflows

• Want to integrate multiple tools

• Require planning and reasoning

• Need systems that can self-improve

Slide 11: Value Proposition

Title: The Business Case

[Visual: ROI metrics for each approach]

MLOps Value:

• 60-70% faster model deployment

• 40-50% reduction in model failures

• 30-40% improvement in model performance

• 3-4x more experiments run

LLMOps Value:

• 80-90% reduction in development time vs custom models

• 50-60% improvement in content quality

• 70% faster iteration on capabilities

• Ability to handle diverse language tasks

Agentic AI Value:

• 40-60% automation of complex workflows

• 24/7 autonomous operation capability

• 30-50% reduction in task completion time

• Ability to handle novel situations

Slide 12: Real-World Adoption

Title: Who's Using What

[Visual: Industry landscape showing adoption patterns]

MLOps Widely Adopted In:

• Financial services (fraud detection)

• Retail (demand forecasting)

• Manufacturing (quality control)

• Healthcare (diagnostics)

LLMOps Growing In:

• Customer service (chatbots)

• Content creation (marketing)

• Legal (document analysis)

• Education (tutoring systems)

Agentic AI Emerging In:

• Research (data analysis)

• Software development (coding assistants)

• Business intelligence (autonomous reporting)

• Personal productivity (assistants)

Slide 13: Measuring Success

Title: Are We Winning?

[Visual: Different dashboards for success metrics]

MLOps Success Metrics:

• Model accuracy & performance

• Time-to-deployment

• Error rates

• Retraining frequency

LLMOps Success Metrics:

• Response quality & relevance

• Hallucination rates

• User satisfaction

• Prompt effectiveness

Agentic AI Success Metrics:

• Task completion rates

• Autonomy level

• Tool usage efficiency

• Error recovery capability

Slide 14: Starting Small

Title: Starting Your Journey

[Visual: Simple starting projects for each approach]

MLOps First Steps:

• Version control for model code

• Experiment tracking

• Basic deployment pipeline

• Simple monitoring

LLMOps First Steps:

• Prompt template management

• Response evaluation framework

• Basic retrieval system

• Output quality checks

Agentic AI First Steps:

• Single-task agent

• Limited tool integration

• Structured task definition

• Human oversight system

Slide 15: Common Misconceptions

Title: Myth vs. Reality

[Visual: Myths being debunked]

MLOps Myth: "It's just DevOps for ML" Reality: Requires specialized practices for data, models, and non-deterministic systems

LLMOps Myth: "Just use the API and you're done" Reality: Requires careful prompt design, evaluation, and retrieval strategies

Agentic AI Myth: "Agents can figure everything out themselves" Reality: Requires careful planning, tool integration, and boundary setting

Slide 16: Future Directions

Title: The Road Ahead

[Visual: Future technology concepts]

MLOps Evolution:

• AutoML integration

• Federated learning operations

• Enhanced explainability

• Edge model deployment

LLMOps Evolution:

• Multimodal operations

• Domain-specific fine-tuning

• Advanced retrieval systems

• Trustworthiness evaluation

Agentic AI Evolution:

• Multi-agent collaboration

• Self-improvement capabilities

• Autonomous tool discovery

• Enhanced planning abilities

Slide 17: Key Skills Needed

Title: Building Your Skill Arsenal

[Visual: Skill badges or development path]

MLOps Skills:

• Data pipeline management

• Model versioning

• CI/CD for ML

• Monitoring & alerting

LLMOps Skills:

• Prompt engineering

• Vector database management

• Evaluation frameworks

• Response filtering

Agentic AI Skills:

• Tool integration

• Task planning

• Memory system design

• Safety guardrails

Slide 18: Asking the Right Questions

Title: Asking the Right Questions

[Visual: Question marks with different themes]

For MLOps Projects:

• How will we track data and model versions?

• What is our retraining strategy?

• How will we monitor model drift?

• What is our deployment process?

For LLMOps Projects:

• How will we manage prompts?

• What retrieval strategy should we use?

• How will we evaluate response quality?

• How do we handle hallucinations?

For Agentic AI Projects:

• What tasks should the agent handle?

• What tools does it need access to?

• How do we ensure task completion?

• What safety measures are required?

Slide 19: Convergence

Title: Where Approaches Converge

[Visual: Venn diagram showing overlap areas]

Shared Fundamentals:

• Version control

• CI/CD pipelines

• Testing automation

• Monitoring systems

• Governance frameworks

The Future: As these fields mature, expect increasing convergence in:

• Tooling ecosystems

• Best practices

• Team structures

• Operational patterns

Slide 20: Bridge to Module 2

Title: The Journey Continues

[Visual: Bridge to next module concept]

What We've Learned:

• The differences between MLOps, LLMOps, and Agentic AI

• When to use each approach

• The value proposition for each

• How to start your journey

Coming in Module 2: A deeper look at the ML and LLM lifecycles - from problem framing to monitoring and feedback loops.

DECK 3: REAL-WORLD IMPACT (20 slides) Slide 1: Introduction to Case Studies Title: Learning from the Pioneers [Visual: Explorers looking at a map] In this section, we'll explore how leading organizations have implemented MLOps, LLMOps, and Agentic AI to solve real business problems. These case studies focus on the "why" and results rather than technical implementation details (which we'll cover in later modules).

Slide 2: Netflix - The Challenge Title: Netflix: Recommendation at Scale [Visual: Netflix interface with recommendation components highlighted] The Business Challenge:

220+ million global subscribers Personalized content for every user Thousands of ML models in production 80% of content views driven by recommendations

The Operational Pain:

Data scientists spending more time on infrastructure than modeling Inconsistent environments between research and production Deployment bottlenecks creating delays Difficult to track which models were in production

Slide 3: Netflix - Metaflow Solution Title: Netflix: The Metaflow Solution [Visual: Simplified Metaflow concept diagram] Metaflow Core Principles:

Human-centered design Seamless local-to-cloud transition Versioning of code, data, and models Focus on data scientist productivity

Key Innovation: Enabling data scientists to develop locally but run at scale without changing their code or workflow. Philosophy: "Make the simple things easy and the hard things possible."

Slide 4: Netflix - Business Impact Title: Netflix: The Results [Visual: Business metrics dashboard with improvements] Business Impact:

60% faster iteration cycles for ML models 70% reduction in time-to-production 4x increase in number of experiments run 80% reduction in deployment-related incidents

Operational Transformation:

From days to hours for model deployment From manual to automated reproducibility From siloed to collaborative data science

Key Lesson: The best MLOps platforms adapt to how data scientists work, not the other way around.

Slide 5: Uber - The Challenge Title: Uber: ML Everywhere [Visual: Uber app with ML touchpoints highlighted] The Business Challenge:

Operating in 10,000+ cities globally ML needed for pricing, ETA, routing, matching 100+ production models requiring constant updates Local models needed for regional optimization

The Operational Pain:

Inconsistent approaches to model deployment Duplicated feature engineering efforts Manual production processes Limited visibility into model performance

Slide 6: Uber - Michelangelo Solution Title: Uber: The Michelangelo Platform [Visual: High-level Michelangelo concept diagram] Michelangelo Approach:

End-to-end ML platform for all Uber's needs Centralized feature store for reusable features Standardized training, evaluation, and deployment Comprehensive monitoring and management

Key Innovation: The feature store - a centralized repository of reusable features that eliminated duplication of engineering efforts.

Slide 7: Uber - Business Impact Title: Uber: The Results [Visual: Before/after metrics visualization] Business Impact:

Model development cycle reduced from months to days 85% of ML workloads running on platform 10,000+ daily batch predictions 70% reduction in ML-related incidents

Operational Transformation:

From siloed feature engineering to shared feature store From manual to automated deployments From limited to comprehensive monitoring

Key Lesson: A unified feature store can be the foundation of scalable MLOps.

Slide 8: OpenAI - The LLMOps Challenge Title: OpenAI: LLMOps at Scale [Visual: ChatGPT interface with behind-the-scenes concept] The Business Challenge:

Creating reliable, safe AI assistants at scale Handling massive inference load globally Ensuring factual accuracy and reducing hallucinations Continuous model improvement from feedback

The Operational Pain:

Traditional ML evaluation metrics didn't apply Prompt management at scale was unprecedented New forms of model failure (hallucinations) Safety and alignment concerns

Slide 9: OpenAI - RLHF Solution Title: OpenAI: The RLHF Approach [Visual: Simplified RLHF concept diagram] RLHF (Reinforcement Learning from Human Feedback):

Human evaluators rate model outputs Ratings used to train reward model Reward model guides model optimization Continuous feedback loop

Key Innovation: Using human preferences to continuously improve model outputs rather than relying solely on objective metrics.

Slide 10: OpenAI - Business Impact Title: OpenAI: The Results [Visual: Quality improvement metrics visualization] Quality Improvements:

40% reduction in hallucination rates from GPT-3.5 to GPT-4 82% improvement in factual knowledge accuracy 63% reduction in harmful content generation 30% improvement in instruction-following

Industry Impact:

Established RLHF as the standard for LLM development Created new operational practices for LLM evaluation Demonstrated the value of human feedback loops

Key Lesson: Human feedback is essential for LLM quality improvement.

Slide 11: Anthropic - Constitutional AI Title: Anthropic: Constitutional AI Approach [Visual: Constitutional principles concept design] The Challenge: Creating safer, more helpful AI assistants Constitutional AI Approach:

Define explicit principles for model behavior Train models to critique their own outputs Use self-critique to improve future outputs Implement human feedback aligned with principles

Key Innovation: Using explicit principles as guardrails for AI behavior, creating more predictable and safer systems.

Slide 12: Emerging Agentic AI: AutoGPT Title: AutoGPT: Agent Autonomy [Visual: AutoGPT conceptual workflow] The Concept: An experimental autonomous agent that:

Takes high-level goals from users Breaks them down into actionable steps Uses tools to accomplish sub-tasks Maintains memory of progress Adapts strategy based on results

Key Innovation: Applying LLM capabilities to autonomous task execution with minimal human intervention.

Slide 13: Agentic AI: Early Applications Title: Agentic AI: Early Applications [Visual: Use cases for agentic systems] Emerging Use Cases:

Research assistance (literature review, data analysis) Content creation (drafting, editing, optimization) Data analysis (exploration, visualization, insights) Business intelligence (report generation, trend spotting) Process automation (multi-step workflows)

Current State: These systems are still emerging but show promising results in controlled environments.

Slide 14: Industry Adoption Patterns Title: Who's Using What: Industry Adoption [Visual: Industry sector map showing adoption rates] Financial Services:

Traditional MLOps for fraud detection (85%) LLMOps for document processing (42%) Agentic AI for market analysis (12%)

Healthcare:

Traditional MLOps for diagnostics (68%) LLMOps for medical research (38%) Agentic AI for clinical workflows (8%)

Retail:

Traditional MLOps for demand forecasting (78%) LLMOps for customer service (65%) Agentic AI for inventory management (15%)

Key Trend: MLOps is mainstream, LLMOps rapidly growing, Agentic AI emerging.

Slide 15: Key Implementation Lessons Title: Lessons from the Frontlines [Visual: Checklist of lessons learned] Lesson 1: Start with Pain Points, Not Technology

Successful implementations address specific operational challenges Focus on measurable outcomes, not tool adoption

Lesson 2: Organizational Buy-in is Critical

Netflix and Uber focused on making tools data scientists want to use Leadership support enabled long-term investment

Lesson 3: Incremental Implementation Works Best

Start with highest-value components Build momentum with early wins before full implementation Evolution beats revolution in operational practices

Lesson 4: Team Structure Matters

Cross-functional teams outperform siloed approaches Bridge roles between data science and engineering are vital Culture is as important as technology

Slide 16: The Business Value of Operational Excellence Title: The Business Bottom Line [Visual: ROI metrics for operational excellence] Quantifiable Benefits:

Speed: 60-70% faster time to production Quality: 30-40% fewer model-related incidents Scale: 3-4x more models in production Innovation: 80% more experiments run

Strategic Benefits:

Competitive advantage through faster innovation Better customer experiences through reliable AI Reduced technical debt and maintenance costs Higher return on ML/AI investments

Slide 17: Common Pitfalls to Avoid Title: Pitfalls to Avoid [Visual: Warning signs with common errors]

1. Starting Too Big

Reality: Big bang approaches usually fail Better Approach: Start small, focused, and iterative

1. Tools Before Strategy

Reality: Tools alone don't solve organizational problems Better Approach: Define processes, then select tools

1. Ignoring Cultural Change

Reality: MLOps requires new workflows for data scientists Better Approach: Focus on adoption and training

1. Neglecting Measurement

Reality: Without metrics, you can't prove value Better Approach: Define baseline and track improvement

Slide 18: Your Self-Assessment Title: Where Are You Today? [Visual: Self-assessment questionnaire] Rate your organization (1-5) on:

Model Versioning: Can you reproduce any model version at any time? Deployment Automation: How automated is your model deployment process? Monitoring: Do you have real-time visibility into model performance? Collaboration: How effectively do data scientists and engineers work together? Data Management: Is your data versioned alongside your models?

This assessment will help you identify your starting point for implementation.

Slide 19: Questions for Reflection Title: Questions to Take With You [Visual: Thought bubbles with key questions]

What is your organization's biggest operational pain point with AI systems? Which case study most closely resembles your situation and challenges? What would a 10% improvement in model deployment time be worth to your business? Are you building for traditional ML, LLMs, agents, or a combination? What's one small, concrete step you could take this week toward operational excellence?

Slide 20: Looking Ahead Title: Your Journey Continues [Visual: Path forward to next module] In This Module:

Understood the MLOps story and evolution Compared MLOps, LLMOps, and Agentic AI approaches Explored real-world case studies and business impact

In Module 2: ML & LLM Lifecycle Overview:

The complete ML lifecycle from problem framing to monitoring How LLMs change the traditional ML workflow Key operational touchpoints for each lifecycle stage Operational requirements throughout the lifecycle

MLOps vs DevOps

Understanding the Evolution

School of DevOps & AI

What is DevOps?

DevOps is a set of practices that combines software development (Dev) and IT operations (Ops) to shorten the development lifecycle and provide continuous delivery of high-quality software.

Key Components:

• Continuous Integration/Continuous Deployment (CI/CD)

• Infrastructure as Code (IaC)

• Monitoring and Logging

• Collaboration and Communication

The DevOps Lifecycle

1. Plan: Define requirements and plan development

2. Code: Write and review code

3. Build: Compile code and create artifacts

4. Test: Automated testing to ensure quality

5. Deploy: Release to production environment

6. Operate: Maintain the system in production

7. Monitor: Track performance and identify issues

8. Feedback: Gather user feedback for improvements

What is MLOps?

MLOps is an extension of DevOps principles applied to machine learning systems, focusing on the reliable and efficient deployment of ML models in production.

Definition:

"MLOps is a set of practices that aims to deploy and maintain machine learning models in production reliably and efficiently."

Why MLOps Emerged

Traditional DevOps wasn't designed to handle the unique challenges of ML systems:

• Data Dependency: ML systems rely heavily on data quality and availability

• Experiment Tracking: ML requires systematic tracking of hyperparameters and metrics

• Model Versioning: Models are complex artifacts that need versioning beyond code

• Model Monitoring: Drift detection and performance degradation monitoring are essential

• Reproducibility: Ensuring consistent model behavior across environments

MLOps vs DevOps: Key Differences

Aspect DevOps MLOps Primary Artifacts Code, Infrastructure Code, Data, Models Testing Focus Functional, Integration Data Quality, Model Performance Versioning Code, Configs Code, Data, Models, Experiments Monitoring System Health, Performance System + Model Drift, Predictions Deployment Application Binaries ML Models + Serving Infrastructure

The Expanded MLOps Lifecycle

1. Data Engineering: Collection, cleaning, and preparation

2. Feature Engineering: Creating features for model training

3. Experimentation & Training: Exploring models and hyperparameters

4. Evaluation: Assessing model performance

5. Versioning: Tracking models, code, and data versions

6. Deployment: Serving models in production

7. Monitoring: Tracking performance and detecting drift

8. Retraining: Updating models with new data

MLOps Maturity Levels

Level 0: Manual Process

• Manual data processing, training, and deployment

• No automation or reproducibility

Level 1: ML Pipeline Automation

• Automated training pipeline

• Continuous training with new data

• Version control for code and models

Level 2: CI/CD for ML

• Automated testing of data, models, and code

• Automated deployment pipelines

• Monitoring in production

Components of MLOps

Infrastructure Layer

• Compute resources (CPU, GPU, TPU)

• Storage for data and models

• Containerization and orchestration

Data Layer

• Data versioning

• Feature stores

• Data validation

Model Layer

• Experiment tracking

• Model registry

• Model serving

Monitoring Layer

• Performance monitoring

• Drift detection

• Alerting systems

Tools Comparison: DevOps vs MLOps

DevOps Tools

• Version Control: Git, SVN

• CI/CD: Jenkins, GitHub Actions, CircleCI

• Infrastructure: Terraform, Ansible, CloudFormation

• Monitoring: Prometheus, Grafana, ELK Stack

MLOps Tools

• Data Version Control: DVC, Pachyderm

• Experiment Tracking: MLflow, Weights & Biases

• Model Registry: MLflow, Neptune

• Feature Store: Feast, Tecton

• Model Serving: TensorFlow Serving, Seldon Core, KServe

Case Study: Traditional Web App vs ML App

Web Application (DevOps)

• Source code in Git

• CI/CD pipeline builds and tests application

• Deployment to staging and production

• Monitoring of system metrics

ML Application (MLOps)

• Source code AND data versions tracked

• CI/CD pipeline includes data validation and model testing

• Feature store for consistent feature engineering

• Model registry for versioning

• A/B testing for model deployment

• Monitoring of both system and model performance

Challenges in MLOps Adoption

1. Skill Gap: Need for expertise in both ML and DevOps

2. Tooling Complexity: Rapidly evolving ecosystem of tools

3. Data Management: Versioning and quality control at scale

4. Reproducibility: Ensuring consistent results across environments

5. Governance and Compliance: Managing model risk and bias

6. Cost Management: Balancing compute resources for training and inference

Best Practices for MLOps

1. Start Simple: Begin with basic automation before complex systems

2. Version Everything: Code, data, models, and configurations

3. Automate Testing: Data quality, model performance, and system tests

4. Monitor Constantly: Track model drift, data quality, and system health

5. Document Thoroughly: Record decisions, architecture, and processes

6. Embrace Collaboration: Break silos between data scientists and engineers

The Future: From MLOps to LLMOps and Agentic AI

Evolution of Operational Practices

• MLOps: Focus on structured data, traditional ML models

• LLMOps: Managing language models, prompt engineering, retrieval systems

• Agentic AI Ops: Orchestrating autonomous agents, tool usage, planning systems

Common Myth: "MLOps is Just DevOps for ML"

The Myth

"MLOps is simply DevOps practices applied to machine learning projects."

The Reality

• MLOps incorporates DevOps principles but extends far beyond them

• ML systems have fundamentally different characteristics:

  • Data dependencies create new failure modes

  • Models require statistical validation, not just functional testing

  • ML systems can degrade silently through concept drift

  • Experiment-driven development differs from feature-driven development

Why This Matters

• Treating MLOps as "just DevOps" leads to gaps in operational readiness

• ML-specific challenges require ML-specific solutions

• Organizations need specialized tools and expertise beyond traditional DevOps

Transferable DevOps Skills for MLOps

DevOps professionals have valuable skills that transfer well to MLOps:

Technical Skills

• Infrastructure as Code: Terraform, Ansible → ML infrastructure provisioning

• Containerization: Docker, Kubernetes → Model packaging and serving

• CI/CD Pipelines: Jenkins, GitHub Actions → Automated model training pipelines

• Monitoring: Prometheus, Grafana → Platform for model monitoring

Soft Skills

• Systems Thinking: Understanding complex system interactions

• Collaboration: Bridging technical gaps between teams

• Automation Mindset: Identifying repetitive tasks for automation

• Incident Management: Responding to and learning from failures

Roadmap: From DevOps to MLOps

1. Build Foundation (1-3 months)

• Learn ML fundamentals (statistics, basic algorithms)

• Understand ML workflow and lifecycle

• Study differences between app development and ML development

2. Acquire ML-Specific Tools (2-4 months)

• Data version control (DVC)

• Experiment tracking (MLflow)

• Model registry and deployment tools

• Feature stores and data validation frameworks

3. Develop Specialized Skills (3-6 months)

• ML pipeline orchestration

• Model monitoring and drift detection

• Performance optimization for ML workloads

• ML-specific testing strategies

4. Gain Practical Experience

• Shadow ML projects in your organization

• Contribute to open-source MLOps tools

• Build a proof-of-concept MLOps pipeline

• Obtain relevant certifications (Cloud ML certifications)

Key Takeaways

1. MLOps extends DevOps to address ML-specific challenges

2. Data and models are first-class citizens in MLOps

3. MLOps requires collaboration between data science and engineering teams

4. Automation and reproducibility are fundamental principles

5. Monitoring goes beyond system metrics to include model performance

6. DevOps skills provide a strong foundation for MLOps

7. The evolution continues as AI systems become more complex

Questions?

Thank you!

School of DevOps & AI

The Emergence of the MLOps Engineer

From DevOps to AI Platform Engineering

School of DevOps

Who does MLOps Afterall ?

• The AI Application Lifecycle

• Evolution of MLOps Roles

• Who Does MLOps?

  • ML Engineers/Data Scientists as MLOps Practitioners

  • DevOps Engineers with ML knowledge

• The Rise of the AI Platform Engineer

• Career Paths in AI Operations

• The Future: ML → LLM → Agentic AI

The AI Application Lifecycle

1. Data Engineering - Collection, preparation, validation

2. Experimentation - Model development, training

3. Operationalization - Deployment, monitoring, governance

4. Iteration - Retraining, optimization, adaptation

Traditional ML Pipeline vs. Production ML System

Research Pipeline:

• Jupyter notebooks

• Local development

• Ad-hoc evaluation

• Manual processes

Production System:

• Automated pipelines

• Scalable infrastructure

• Continuous monitoring

• Governance & compliance

The MLOps Gap

The Challenge:

• 87% of ML projects never make it to production

• Data scientists lack infrastructure expertise

• DevOps teams lack ML knowledge

• No standardized practices for operationalizing ML

Evolution of MLOps Roles

MLOps Level 0: Manual processes, no automation

• "It works on my machine!"

MLOps Level 1: ML pipeline automation, manual deployment

• Automated Training and Data Pipelines

MLOps Level 2: Automated CI/CD

• Full automation of ML Deployments

MLOps Level 3: Automated retraining pipelines

• Full automation of ML lifecycle

Who Does MLOps?

Two Organizational Approaches:

1. End-to-End Data Scientists (Full Stack)

   • Single individuals handling entire ML lifecycle

   • From data to deployment to monitoring

   • "Renaissance" professionals with broad skillsets

2. Cross-Functional Teams

   • Specialized roles collaborating on ML systems

   • Clear ownership boundaries

   • Shared responsibility for production success

The End-to-End Data Scientist Approach

Advantages:

• No handoffs between teams

• Faster iteration cycles

• Full context on model development and operation

• Ownership of entire ML lifecycle

Challenges:

• Rare skill combination (unicorn hunting)

• Time spent on infrastructure vs. core modeling

• Difficult to scale across multiple projects

• Risk of non-standard implementations

• Burnout from context switching

Cross-Functional Team Approach

Key Roles:

• Data Scientists: Model development, evaluation

• ML Engineers: Model optimization, pipeline development

• Data Engineers: Data pipelines, feature stores

• MLOps/Platform Engineers: Infrastructure, CI/CD, monitoring

• Software Engineers: Application integration, APIs

Benefits:

• Specialized expertise at each stage

• Scalable across multiple ML initiatives

• Standard practices across projects

• Reduced single-person dependencies

The Buy vs. Build Decision

When to Buy a Platform:

• End-to-End Data Scientists: Need comprehensive tooling

• Limited infrastructure expertise in-house

• Early ML maturity stages

• Focus on rapid time-to-market

• Standard ML workflows with common patterns

When to Build:

• Cross-Functional Teams: Can create tailored solutions

• Sufficient scale to justify investment

• Unique requirements not met by vendors

• Data security/compliance requires custom solutions

• Advanced ML maturity with specialized workflows

Roles in Cross-Functional Teams

Two Key Operational Tracks:

1. MLOps Practitioners

   • Data Scientists with operational knowledge

   • ML Engineers who productionize models

   • Focus: Model quality, experiment tracking, evaluation

2. MLOps Engineers/AI Platform Engineers

   • DevOps Engineers specialized in ML infrastructure

   • Platform Engineers building reusable components

   • Focus: Scalable infrastructure, automation, governance

The Rise of the AI Platform Engineer

Evolution of the MLOps Engineer Role:

• From: Managing traditional ML workflows

• To: Building platforms for all AI workloads

  • Traditional ML models

  • Large Language Models (LLMs)

  • Generative AI

  • Agentic AI systems

Key Responsibilities:

• Design unified platforms for all AI workloads

• Create standardized deployment patterns

• Establish governance frameworks

• Enable self-service for data scientists and ML engineers

MLOps to LLMOps to AgenticAIOps

Traditional MLOps:

• Data/model versioning, automated pipelines, monitoring

LLMOps Additions:

• Prompt management, vector databases, RAG architectures

AgenticAIOps:

• Tool integration, agent orchestration, feedback systems

Skills Matrix

Role Technical Skills Domain Knowledge MLOps Practitioner Python, ML frameworks, basic Docker Strong ML, statistics MLOps Engineer Kubernetes, Terraform, CI/CD, monitoring Basic ML understanding AI Platform Engineer Advanced K8s, cloud platforms, security Broad AI knowledge

Evolution of Team Structures

Stage 1: Single Expert (Startup Phase)

• End-to-end data scientist handling everything

• Buy off-the-shelf ML platforms

• Limited scale, focused use cases

Stage 2: Specialized Team (Growth Phase)

• Dedicated ML/AI team with specialized roles

• Mix of bought platforms and custom components

• Multiple ML use cases in production

Stage 3: Platform Team (Enterprise Scale)

• Central AI Platform team supporting multiple ML teams

• Custom platforms with reusable components

• Dozens/hundreds of models in production

The AI/ML Dream Team 2025

Core Roles in Cross-Functional Teams:

• Data Engineer: Data pipelines, quality, storage

• Data Scientist: Analysis, model development, evaluation

• ML/AI Engineer: Model architectures, optimization, embedding

• MLOps Specialist/AI Platform Engineer: Infrastructure, CI/CD, monitoring

Career Paths:

• Path 1: Data Scientist/Software Enggg/Data Engg → ML Engineer → MLOps Practitioner

• Path 2: DevOps Engineer → MLOps Engineer → AI Platform Engineer

Working together to deliver reliable, scalable AI systems

School of DevOps AI Education Tracks

Track 1: MLOps Practitioner

• For: Data Scientists, ML Engineers, Software Engineers, Data Engineers

• Focus: Operationalizing models, monitoring performance

Track 2: AI Platform Engineer

• For: DevOps Engineers, Infrastructure Specialists

• Focus: Building platforms to support all AI workloads

Key Takeaways

1. MLOps bridges the gap between ML research and production

2. Two converging paths: ML + DevOps and DevOps + ML

3. AI Platform Engineering is emerging as ML evolves to LLMs and Agentic AI

4. Organizations need both practitioners and platform builders

5. School of DevOps offers specialized training for both paths