Udemy
Turn what you know into an opportunity and reach millions around the world.
Learn More
Your cart is empty.
Keep shopping
AI Threat Modeling: Secure LLM, RAG and Agent Systems
New
102 students

AI Threat Modeling: Secure LLM, RAG and Agent Systems

Map, rank and test the threats in chat assistants, RAG pipelines and AI agents, with recorded labs in free tools
Last updated 11/2026
English
English

What you'll learn

  • Draw a data-flow diagram of an AI system and mark the trust boundaries that only exist in AI
  • Use the OWASP Top 10 for LLM Applications, MITRE ATLAS and NIST AI 100-2 as one working threat catalog
  • Apply STRIDE per element to models, prompts, retrieval, tools and agent memory
  • Find prompt injection, sensitive information disclosure and improper output handling as design flaws, before code ships
  • Threat-model a RAG pipeline, including poisoned documents and access control on retrieved data
  • Threat-model an AI agent with tools: excessive agency, confused deputy, tool poisoning and memory manipulation
  • Rank AI threats with a simple risk score and record accept, mitigate and transfer decisions
  • Choose controls, understand what guardrails miss, and turn each threat into a test case
  • Test controls with promptfoo against a running demo app and keep the evidence
  • Produce a threat model report and keep it current as models, tools and data sources change

Course content

9 sections • 43 lectures • 4h 44m total length
  • Welcome: What We Will Build5:01
  • What Makes AI Systems Different6:26
  • Where Threat Modeling Fits6:41
  • The Four Questions, Applied to AI6:15
  • Quiz 1 — Why AI systems need their own threat model

Requirements

  • Basic security knowledge, such as what authentication, access control and injection attacks are
  • A general idea of how chat assistants and LLM-based applications work; no machine learning mathematics
  • A computer that can run free tools such as OWASP Threat Dragon, Python and Node.js if you want to repeat the labs

Description

This course contains the use of artificial intelligence.

Most AI security material either lists risks or teaches attacks. This course teaches the step in between: the design-time method that turns an AI system into a diagram, a ranked threat list, chosen controls and tests that prove those controls work.

One system, three stages. We follow the Northwind Shop Assistant, a fictional online retailer's shopping and support assistant. It starts as a chat assistant on a hosted model, gains retrieval over the product catalog, reviews and order history, and ends as an agent that can issue refunds, apply discount codes and email customers through tools. Each stage adds attack surface, so the threat model grows lecture by lecture, the way real systems do.

What the course covers

  • Data-flow diagrams for AI systems, and the trust boundaries that only exist in AI: system prompt, user input, retrieved documents, tool results and model output
  • The OWASP Top 10 for LLM Applications, MITRE ATLAS, NIST AI 100-2 and the OWASP agentic guidance, mapped into one working threat catalog
  • STRIDE per element adapted for AI components, prompt injection as a trust-boundary failure, excessive agency, supply chain and denial of wallet
  • RAG threats: poisoned documents, retrieval manipulation and access control on retrieved data
  • Agent and tool threats: confused deputy, over-broad scopes, tool poisoning, memory manipulation, and human-in-the-loop controls that actually work
  • A simple AI risk score, written risk decisions, controls, guardrails and their limits, and turning threats into test cases
  • Keeping the threat model current in the delivery pipeline, and a report an auditor or customer will accept

Seven recorded labs, all with free tools. We diagram in OWASP Threat Dragon, write threats as code with pytm, apply the CSA MAESTRO layers to the agent, rank the threat list in a spreadsheet, and test three controls with promptfoo against a small demo app you can run yourself. The capstone puts the full model, the report and the test evidence together.

What you get

  • 43 video lectures with captions, about 4 hours 45 minutes in total
  • A quiz at the end of each of the 9 sections, a 40-question practice test and 3 practical assignments
  • Downloads: an AI threat catalog, a risk scoring sheet, a threat model report template and a worked example, the Threat Dragon models, the pytm script, the promptfoo configuration and the demo app source

The course assumes basic security knowledge. No machine learning mathematics and no paid tools are needed.

Who this course is for:

  • Security engineers, application security and product security teams reviewing AI features
  • AI and machine learning engineers who build LLM, RAG or agent systems
  • Security architects who must sign off designs that include AI components
  • Governance, risk and compliance professionals who need evidence that AI risks were assessed