AI Security & AI Guardrails : Agents, Red Teaming & Hands-On

This section is all theory, with no code. Already comfortable with LLMs, embeddings and RAG? Skip ahead to Section 3. New to GenAI? Stay for about 40 minutes: every guardrail later in the course builds on these ideas.

Foundation model, language model, LLM, multimodal model — they're nested categories, not synonyms, and this untangles which is which.

Clear definitions of foundation models, language models and LLMs, and how they relate. You will learn how foundation models are trained, what general-purpose training means, and how fine-tuning adapts a model to a task.

Clear definitions of foundation models, language models and LLMs, and how they relate. You will learn how foundation models are trained, what general-purpose training means, and how fine-tuning adapts a model to a task.

Inference parameters are the settings behind that inconsistency — this video builds the vocabulary (temperature, top-K, top-P, length control) before putting real numbers behind them in part two.

Why the same prompt can return a safe answer one time and a wild one the next. Learn the vocabulary of randomness and diversity: temperature, top-k, top-p, response length and stop sequences

One example, run through Temperature, Top-K and Top-P stacked together — and why the settings that make output more predictable are the same ones that make it easier to guard.

How embedding models turn text into vectors so a search for "affordable car" finds "cheap automobile". This is the basis of RAG, semantic filters and several evaluators later in the course.

The essential components of a prompt (instruction, context, input data and output format) and how they fit together. Knowing the anatomy of a prompt is what makes prompt injection easy to understand in Part 2.

How attackers override an assistant's instructions, illustrated by a shopping assistant manipulated into favouring one product. Covers direct and indirect injection and why ordinary filtering misses them.

Meta's Prompt Guard is a small classifier that screens every input before it reaches your model. Learn the attack vectors it detects and the strategies for deploying it.

Run Prompt Guard in Google Colab: request gated Hugging Face access, set up a GPU runtime, and score benign and malicious prompts. You need a Hugging Face account and approved model access.

How Llama Guard classifies a conversation as safe or unsafe against hazard categories, and how its prompt template works. This merges the former "Llama Guard Theory" and "Llama Guard Prompt" lectures and includes the Llama Guard 4 update.

Moderate safe and unsafe prompts with Llama Guard in Colab using its chat template. The 8B model needs a high-memory GPU runtime (a paid Colab tier); the lecture shows what to expect.

A quick preview of this section — how attackers hide instructions inside images, and why a model that can look at a picture needs guardrails a text-only model never did.

A blank image that makes a vision model advertise a sale: how attackers hide instructions in pixels, why this is dangerous for multimodal apps, and the defenses that work.

The Llama Guard 3 family (1B, 8B and 11B Vision), its hazard categories, special tokens, prompt template, and how to customize or exclude categories. Also covers the model's limitations with images.

Orientation to the section through a well-known public AI error, and what you will be able to detect by the end.

What hallucination is, why it happens (data quality and grounding problems), the main types, and the practices that prevent it.

How to detect hallucination automatically: faithfulness to the source, answer relevance, specialized detection models, and practical techniques.

Use Phi-3 as a yes/no hallucination judge on consistent and inconsistent answers, and see where the judge succeeds and where it fails. Code: code/phi3_hallucination_code.py.

Score premise/hypothesis pairs for factual consistency with vectara/hallucination_evaluation_model.
Notebook: code/vectara_hallucination.ipynb.

Orientation to the section through a real case of employees pasting confidential code into a public chatbot.

What Presidio is, why LLM applications need data-loss prevention, the problems it solves, and how it maps to OWASP LLM02.

Presidio's four components and the end-to-end data flow: find sensitive data first, then hide it.

Inside the Analyzer: recognizers, NLP engines, context words and confidence scores. Includes why "Jordan" can be a person or a country.

Run the Analyzer on real text, read its entity results and scores, and tune the confidence thresholds.

Add your own entity type (for example, an employee or policy number) with a custom recognizer, and test it.

Anonymizer operators (replace, redact, mask, hash and encrypt), and how to use Presidio as a guardrail around an LLM call.

Chain the Analyzer and Anonymizer to clean a prompt before it is sent to an LLM. Note: use a Luhn-valid test card number so Presidio detects it.

From DEF CON's public AI red-teaming event to your own app: where the term comes from, why AI needs it, what red teamers target, and the red-teaming loop.

Garak, the open-source LLM vulnerability scanner: the vulnerabilities it tests for, its key features and its architecture.

Point Garak at a real model and run a complete scan start to finish, then read the report it produces.

How probes test a model for a specific failure mode, and how detectors decide pass or fail.

Install Garak, walk through the command line, and run an end-to-end scan.

Probe whether a model can be tricked by encoded text (Base64 and others) that slips past input filters.

What data exfiltration is, and how attackers use an LLM to move sensitive information out of a secure system.

Run Garak's exfiltration probes and interpret the results.

Use Garak's profanity (LMRC) probes to test whether the model can be pushed into producing abusive language.

Why evaluation matters, illustrated by an airline chatbot whose answer cost the company in court. Covers the types of evaluation, model-based evaluators, LLM-as-judge vs. small models, and the faithfulness, SAS and context-relevance metrics.

The runtime flow of an evaluator: inputs, ground truth, scoring and reporting.

Use an LLM-as-judge faithfulness evaluator to score whether answers are supported by the retrieved context.

Measure whether the retrieved context is relevant to the question.

Measure semantic answer similarity (SAS) against ground truth, and whether the retrieved context is relevant to the question.

Agents go beyond answering: they take action. Covers agents, tasks, components, build time and the runtime stages.

Follow a single request ("cancel order 1234") through the agent loop of reasoning, tool calls and observations, and see where it can break.

Penetration testing basics and common web vulnerabilities (XSS, SQL injection) that the agent will look for.

What you will build: an agent that runs a security scan and returns a report. Covers the goal, the loop, the tools and the rules it must follow.

CrewAI's agents, tasks, tools and crews: the framework used for this project.

Run a manual scan with the OWASP ZAP proxy so you know what the agent will automate.

Wrap ZAP as a tool the agent can call, and test the tool call.

Define the CrewAI agent, then run an automated penetration test.

Run the agent end to end and evaluate the quality of its findings and report.

Split the work across several cooperating agents for a deeper scan.

Evaluate the multi-agent run and compare it with the single-agent result.