AI Security & Governance Masterclass: Build, Attack & Defend

Explore how ai security differs from traditional application security, noting prompts, documents, and chat history shaping model behavior and trust boundary, with layered guardrails across input, model, tools, and output.

Identify the OWASP top 10 risks for LLM applications, including prompt injection and sensitive information disclosure, and map them to practical defenses across the full app.

Adopt a layered AI security approach using multiple controls along a secure pipeline. From prompt validation to context isolation and tool security, every layer reduces risk and provides visibility.

Build a Streamlit AI chat assistant by wiring local Ulama and OpenAI backends through a configurable service, config, and prompts, ensuring testable, portable deployment.

Explore direct prompt injection, where a user message tries to override an ai assistant, and learn to apply system, developer, and user instruction hierarchy with prompt validation and guardrails.

Explore prompt hierarchy and how system, developer, and user messages determine authority in secure AI applications. Learn defenses like validation, guardrails, and monitoring to prevent prompt injection.

Explore prompt injection techniques on a chat assistant, exposing how hidden system prompts can be breached. Implement prompt validation to secure AI conversations.

Add a basic guardrail layer to block prompts with a safe, consistent refusal before the model, and implement a guardrails file with a safe refusal function for reuse.

test prompt injection defenses by running the app in a virtual environment, validating prompts with guardrails to block risky inputs before the model processes them, demonstrating a first defense layer.

Explore how ai jailbreaks push models beyond safety boundaries, analyze patterns like roleplay and prompt injection, and build defense with guardrails, validation, and monitoring.

Explore role-playing, encoding, and multi-turn attacks that hide intent and bypass model boundaries, and learn layered defenses, including validation, guardrails, and adversarial testing to secure conversations.

Demonstrate jailbreak attacks against a chat assistant and test prompt validation and guardrails. Preview the next lab, building a prompt risk analyzer to classify risky prompts before responses.

Add a prompt filtering layer atop the risk analyzer to convert risk scores into allow, warn, or block decisions, with high-risk prompts blocked and medium-risk prompts warned.

Evaluate jailbreak resistance through a multi-layer defense workflow that analyzes prompt risk, filters prompts, and blocks high-risk inputs while preserving safe educational prompts.

Retrieval augmented generation retrieves relevant documents before answering, using embeddings and vector search to provide grounded, source backed answers, while highlighting security risks and defenses.

Explore how rag applications work by retrieving from a knowledge base, adding relevant content to prompts, and generating grounded answers with embeddings and vector search, with a security focus.

Design and implement a document-based rag assistant from scratch, exploring security and governance considerations in building reliable, accountable ai retrieval systems.

Explore retrieval augmented generation by querying uploaded documents and verifying answers stay within scope, while learning safe mitigation and defense against rag attacks.

Examine prompt injection risks in retrieval augmented generation, where retrieved content influences model behavior, and learn defenses like trusted source validation and content isolation.

Explore how retrieval augmented generation can be hijacked by poisoned documents and learn to enforce trust boundaries through source validation, context isolation, and content sanitization.

Expose security risks in a RAG assistant by ingesting a malicious PDF, revealing prompt injection through retrieved content, and outlining defenses like trusted source validation, context isolation, and sanitization.

Examine how a poisoned retrieved context can mislead an AI assistant, and learn defenses like trusted source validation, metadata checks, context isolation, and sanitization.

Add context isolation to the rag assistant by wrapping retrieved content with start and end markers, treating untrusted document text as reference data and separating instructions from data.

Apply content sanitization to retrieved text in the rag pipeline, neutralizing risky phrases and fake labels before context is passed to the model, using a sanitizer module and cleanup rules.

Verify the RAG security defenses by testing trusted source validation, context isolation, and retrieve context sanitization, and observe how untrusted content is blocked.

Learn how tool calling enables ai assistants to use external functions through a secure, registry-driven workflow; validate inputs, enforce permissions, and connect natural language requests to tools.

Explore function calling and tool selection, where the model proposes tool calls but the application validates, approves, and executes them safely using a structured tool request and registry.

Design three assistant tools: weather, calculator, and email—and learn secure tool calling through structured inputs, validation, permissions, logging, and human approval to ensure safe external actions.

Test multiple tool calls in the tool‑calling chat assistant, verify models and tools (calculator, weather, email draft), and illustrate end‑to‑end workflow from input to execution.

Understand the security risks of tool calling and how the application enforces policy, validates inputs, limits permissions, and requires human review before sensitive actions.

Explain parameter injection in tool calls and why validation, allow lists, and permissions—plus human approval for high‑impact actions—prevent unauthorized actions and unknown fields.

Examine weaknesses of a tool calling chat assistant by testing the email draft tool for unauthorized use, highlighting missing authorization, prompt injection risks, and the need for human approval.

Implement a central input validator that sits between the registry and tools, enforcing schema fidelity, safety checks, and injection protection to guard tool execution.

Implement a permission layer for tool calls that classifies tools as read or action, enforces policy in the registry, and makes configuration-driven, environment-aware approvals central.

Introduce a human approval gate for sensitive tool actions, ensuring the assistant proposes actions but requires explicit user approval before execution.

Explore memory poisoning attacks and how stored memory can influence future conversations and trusted context. Apply defenses: validate memory before storage, require approval, isolate memory, and log changes.

Persistent context enables cross-session memory but creates risk of memory poisoning. Design a governed memory system with provenance, scope, approval, expiry, and audit to prevent unsafe data influencing actions.

Implement a memory validator to gate writes before storage, filtering oversized or instruction-like inputs with regex checks and a length cap to prevent memory poisoning.

Introduce a human approval gate before memory storage, enforcing an approval policy and pending memory flow via the service layer and memory store, with an approval prompt pausing saves.

Demonstrates a memory security workflow that validates memories, blocks unsafe ones, and requires user approval for safe and gray zone memories, ensuring all memory operations pass validation first.