AI Security Masterclass: Prompt Injection & LLM Security

This course builds a secure artificial intelligence application in layers, from a chat assistant to prompt security, ROG, and RAG security, memory, tools, and agents, culminating in a security gateway.

Outline the course roadmap from foundations to prompt security, RAG, tool calling, memory, and agents. Build, attack, and secure AI applications, culminating in a production-grade security gateway and capstone.

Install and verify Python, Visual Studio Code, and Git to set up a clean development environment and confirm Python (or Python 3), pip, VS Code, and Git work.

Map the OWASP top 10 risks for LLM applications, including prompt injection, sensitive information disclosure, data poisoning, and embedding weaknesses, and outline defenses across data, prompts, tools, and vectors.

Build a working AI chat assistant in Streamlit, wiring local Ulama and OpenAI as providers, with a layered config, system prompt, and service architecture for scalable testing.

Explain how prompt injection techniques test a chat assistant, reveal hidden system instructions, explore attack vectors, and emphasize prompt validation and guard rate concepts for secure llama interactions.

Implement a prompt validation gate that blocks jailbreak phrases and hidden instruction attacks before the model sees the message, using a block-pattern check and a safe block message.

Add a basic guardrail layer to block prompts with a safe, consistent refusal before reaching the model. Centralize safety logic to ensure consistency and prepare for expanding guardrails.

Test prompt injection defenses by running a local app in a virtual environment, validating prompts, and observing guardrails block risky inputs before reaching the model.

Explore AI jailbreaks and prompt injection, define how attackers press model boundaries through roleplay, framing, and language tricks, and learn app-level defenses, guardrails, and defense testing.

Explore role-playing, encoding, and multi-turn manipulation as attack techniques that hide intent, bypass safeguards, and exploit framing and context across conversation; learn robust defenses and layered validation.

Examine jailbreaks against a chat assistant, test attack prompts: role-playing, encoding, and multi-turn tricks, and study guardrails, prompt validation, and the upcoming prompt risk analyzer to strengthen defenses.

Build a prompt risk analyzer that scores jailbreak signals like roleplay, encoding, and hidden instructions, with base64 checks, integrated into the chat app to reveal risk before responding.

Add a prompt filter atop the risk analyzer to convert risk results into allow, warn, or block decisions and connect this policy layer to app.py.

Test jailbreak resistance demonstrates a layered defense with a risk analyzer and prompt filter that score prompts by category (instruction override, hidden instruction extraction) and risk, blocking high-risk inputs.

Master retrieval augmented generation by retrieving relevant documents before answering, enriching the model context with chunked content and embeddings for grounded, accurate responses, with security considerations.

Explore how retrieval augmented generation uses semantic embeddings and vector search to provide grounded answers from retrieved company documents and manuals with cited sources.

Explore how embeddings convert text into meaning, enabling vector search to retrieve relevant chunks that ground LLM answers and RAG systems, with chunking, similarity, and security considerations.

Build a document-based RAG assistant from scratch, focusing on prompt injection and LLM security in the AI security masterclass.

Demonstrates a retrieval augmented generation app that answers questions from uploaded documents, tests inside vs outside scope, and introduces prompt injection with defenses.

Add context isolation to the RAG assistant by wrapping retrieved content with start and end markers, treating untrusted data as reference material.

Implement content sanitization in the rag assistant to neutralize risky retrieved text before prompting, using a sanitizer module that normalizes Unicode, removes zero-width characters, and redacts injection phrases.

Test the RAG defenses by loading documents and verifying trusted source validation, context isolation, and retrieve context sanitization; ensure prompt injection detection is active and trusted content is used.

Explore tool calling, enabling AI to use external tools like weather and calculator through a secure five-step flow, with tool registries, input schemas, and strict permissions for safe operation.

Demonstrate secure tool calling in the ai security masterclass with three tools, weather, calculator, and email, using schemas, input validation, permissions, and audit logging from design to implementation.

Explore tool calling security risks and learn how applications enforce policy, validate parameters, and apply least privilege to prevent unsafe tool use and sensitive actions.

Explore weaknesses in tool calling chat assistants, focusing on prompt injection and unauthorized tool calls. Analyze missing authorization, approval workflows, and policy separation to secure tool execution.

Implement a permission layer for a tool calling assistant by classifying tools as read or action level and enforcing policy through a registry and a permission checker.

Demonstrates verifying tool security defenses in an app by blocking unsafe inputs, requiring approval for sensitive tools, and validating parameters to ensure policy compliance while preserving safe tool use.

Differentiate short-term memory and long-term memory in an ai assistant, explaining how context, memory policy, and secure retrieval balance usefulness with privacy and security.

Design persistent AI memory with a structured memory schema and a life cycle that includes detection, validation, user approval, retrieval, update, and deletion, ensuring security and auditability.

Build a persistent memory module that stores facts locally in a json file and feeds remembered context into future assistant responses.

Test memory retrieval demonstrates saving an approved user preference, like preferring python examples, and making memory visible and controllable, with retrieval that informs personalized responses.