Securing Agentic AI: Attack & Defend Autonomous Agents

Hands-on agent security: prompt injection, tool abuse, least privilege, sandboxing & red-teaming with LangGraph/MCP

New

Created byDr. Amar Massoud

Last updated 7/2026

English

English [Auto],

What you'll learn

Map the attack surface of any agentic workflow: tools, memory, plan loop, multi-agent edges
Exploit prompt injection (direct & indirect), tool/command/SSRF injection, memory poisoning
Hijack agent plans and trigger denial-of-wallet; defend with execution guards
Enforce least privilege with permission brokers, MCP scopes, and approval gates
Secure multi-agent systems (CrewAI/AutoGen) against agent-to-agent injection & confused-deputy
Sandbox code execution with containers + egress control; defend the tool/MCP supply chain
Instrument tracing, build runtime guardrails, and run automated red-teaming (Garak, PyRIT)
Write a governance-grade red-team report mapped to OWASP, ATLAS, NIST, EU AI Act, ISO 42001

Course content

9 sections • 46 lectures • 3h 34m total length

Welcome — Why Securing Agents Is a New Discipline5:24
Discover why securing agentic AI is a new discipline, centered on action amplification and multi-tool agents. Map attack surfaces and apply OWASP, Atlas, and NIST for defense and remediation.
Anatomy of an Agent — The Perceive-Plan-Act Loop5:36
The Agent Attack Surface — Where Trust Breaks5:49
Threat Frameworks — OWASP, MITRE ATLAS, NIST AI RMF5:38
Meet TaskPilot — Your Vulnerable Lab Agent4:56
Set Up Your Lab Environment (macOS · Windows · Linux)5:41
? Download the lab kit (securing-agentic-ai-labs.zip) from the Resources section of this lecture, then unzip it and follow along.
Lab 1 — Build & Map a Vulnerable Agent (TaskPilot)5:00
Section 1 Quiz — Foundations of Agent Security
Threat-Model an Agentic Workflow's Attack Surface

Direct Prompt Injection — Hijacking the Instruction Channel4:51
Direct prompt injection hijacks user input to override the system prompt and trigger tool calls. Learn causes, steps, and defenses—spotlighting, delimiting, least-privileged tool scopes, approvals, and detection in Task Pilot.
Indirect Injection — When Tool Output Becomes a Command4:15
RAG & Memory Poisoning — Persisting the Payload5:04
Defenses I — Spotlighting, Delimiting & Dual-LLM Patterns4:49
Layer defenses around a core principle: untrusted data is never an instruction. Apply spotlighting, delimiting, and dual-llm patterns to separate data, enforce schemas, and isolate task pilot from untrusted text.
Defenses II — Output Handling & Injection Detection5:15
Lab 2 — Indirect Prompt Injection & Defenses4:02
Section 2 Quiz — Prompt Injection in Agent Loops
Build a 3-Layer Prompt-Injection Defense for an Agent Loop

Secure Tool Design — The Tool Is Your Trust Boundary4:58
Treat the tool boundary as a trust boundary between an untrusted model and the real world. Enforce a four-step pattern: narrow schema, boundary validation, least privilege, and output logging.
Argument & Command Injection in Tools4:40
Trace how a tool argument becomes a shell command in TaskPilot, revealing metacharacter injection and defending with no shell and parameterized APIs.
SSRF, Path Traversal & the Fetch and File Tools5:23
MCP Server Security — Schemas, Descriptions & Trust5:51
Secure MCP server integrations by vetting and pinning servers and applying least privilege for tool access. Tighten tool schemas and descriptions to prevent prompt injection, tool shadowing, and data exfiltration.
Hardening TaskPilot's Tools4:50
Lab 3 — Tool Exploitation → Hardening3:50
Section 3 Quiz — Tool-Calling Vulnerabilities
Redesign an Agent Tool into a Secure Trust Boundary

Plan & Goal Hijacking — Steering the Multi-Step Loop5:10
Trace how an attacker hijacks an agent's plan in a multi-step loop by injecting poisoned observations and overriding goals, and review defenses such as pinned objectives and guard checks.
Memory & State Poisoning Across Sessions5:17
Learn how memory poisoning creates durable attacker-controlled instructions across sessions and how to defend by validating on write, sanitizing on read, and scoping memory.
Loop, Recursion & Cost Abuse — Denial of Wallet4:16
Trace how a normal agent loop degrades into a self-fueling, cost-draining cycle and learn four defenses: terminate the loop, track cost, measure progress, and escalate to SAM or an operator.
Execution Guards — Step Limits, Budgets & Checkpoints4:48
Lab 5 — Execution & Memory Guards3:26
Section 5 Quiz — Multi-Step Execution & Memory Security
Add Execution Guards and Memory Provenance to a Multi-Step Agent

Agent-to-Agent Injection — Trust Between Agents4:26
The Confused Deputy & Malicious Delegation4:20
Orchestration Risk in CrewAI & AutoGen4:31
Examine orchestration risks in Crew AI and AutoGen across four layers—the orchestrator, role-named agents, shared context, and tools—and apply four defenses: least privilege, untrusted data, max limits, and a human-in-the-loop.
Human-in-the-Loop Gates for Autonomy4:57
Explore human-in-the-loop gates that pause risky actions, route irreversible calls to a human reviewer, and log every decision in an immutable audit trail.
Lab 6 — Multi-Agent Attack + Human-in-the-Loop3:34
Section 6 Quiz — Multi-Agent & Autonomous Systems
Threat-Model a Multi-Agent Crew's Trust Boundaries

Tracing & Observability for Agents with OpenTelemetry4:44
Runtime Guardrails — Input, Output & Action Filters5:18
Automated Red-Teaming — Garak & PyRIT4:46
Governance — NIST AI RMF, EU AI Act, ISO 420014:46
Explore automated red teaming for agentic AI governance with Garrick and PYRIT, mapping findings to OWASP LLM01/LLM06 and MITRE Atlas, aligned with NIST AI RMF, EU AI Act, ISO 42001.
Incident Response & the Capstone Red-Team Report5:03
Lab 8 (Capstone) — Defend & Report3:18
Section 8 Quiz — Detection, Monitoring & Governance
Capstone — Produce a Client-Ready Agent Red-Team & Remediation Report
Final Exam — Securing Agentic AI (30 Q, 70% pass)

Requirements

Python fluency; familiarity with OWASP-style security frameworks
Sandbox code execution with containers + egress control; defend the tool/MCP supply chain

Description

This course contains the use of artificial intelligence.

Autonomous AI agents introduce entirely new attack vectors. The moment an LLM can call a shell, read a database, fetch a URL, send email, and run code across multiple steps, a single manipulation stops being unsafe text and becomes an executed action — at scale. This is action amplification, and it's why securing agents is its own discipline.

This is a practical, attack-then-defend course. You'll build a deliberately vulnerable agent — TaskPilot — on LangGraph and a local Llama model, then attack and harden it across eight hands-on labs. You'll exploit direct and indirect prompt injection, tool and command injection, SSRF, memory poisoning, plan hijacking, denial-of-wallet, agent-to-agent injection, and sandbox escape — then build the defenses: spotlighting and output handling, secure tool design, least-privilege permission brokers, MCP scoping, execution guards, container sandboxing with egress control, human-in-the-loop gates, tracing, runtime guardrails, and automated red-teaming with Garak and PyRIT.

Everything runs locally on a 16 GB laptop with Ollama — no paid API keys, no cloud accounts. Every lecture and lab is anchored to the frameworks that matter: the OWASP Top 10 for LLM Applications and the OWASP Agentic Security Initiative, MITRE ATLAS, the NIST AI Risk Management Framework, the EU AI Act, and ISO/IEC 42001. The capstone is a publishable red-team and remediation report you can adapt for client engagements.

Who this course is for:

Security engineers, red-teamers, AppSec leads, and AI/platform engineers building or defending agents
Instrument tracing, build runtime guardrails, and run automated red-teaming (Garak, PyRIT)

Securing Agentic AI: Attack & Defend Autonomous Agents

What you'll learn

Explore related topics

Course content

Foundations of Agent Security7 lectures • 38min

Prompt Injection in Agent Loops6 lectures • 28min

Tool-Calling Vulnerabilities6 lectures • 30min

Excessive Agency & Least Privilege5 lectures • 22min

Multi-Step Execution & Memory Security5 lectures • 23min

Multi-Agent & Autonomous Systems5 lectures • 22min

Sandboxing, Isolation & Secure Execution5 lectures • 23min

Detection, Monitoring & Governance6 lectures • 28min

Bonus: Free Resource1 lecture • 1min

Requirements

Description

Who this course is for: