
Explore an overview of AI attack categories, including misalignment, jailbreaks, and prompt injections, and understand how injections, data exfiltration, and plugins create security risks in LLMs.
AI LLM01: Prompt Injection - Theory Part 1
AI LLM01: Prompt Injection - Theory Part 2
Explore prompt injection beyond text, including direct and indirect methods via images and audio. See an image-based demo of embedded instructions and steganography.
Explore indirect prompt injection in a vulnerable recruitment app, demonstrating how a prompt embedded in resumes or emails can exfiltrate base64 encoded data and pii during pre-screening.
Explore how multimedia content can trigger prompt injection in LLMs. The video-based jailbreak demonstrates bypassing guardrails and highlights multimedia threats for penetration testing and defense.
Examine how ANSI escape codes in terminal emulators enable prompt injection in LLMs, revealing risks such as beeps, cursor moves, hidden text, clipboard exfiltration, and potential remote code execution.
Explore MCP server vulnerabilities in agentic AI via image prompt injection that triggers tool invocation on a cloud desktop, and learn how indirect image prompts enable automated calls.
Prompt Injection in Skills
Indirect Prompt Injection in Grok
ASCII Unicode Prompt Injection into Google Calendar. DoS and Phishing
Indirect Prompt Injection via User Agent ending up in log files
Apple Mail Summary Prompt Injection
ChatGPT Gmail and Calendar Connectors - Prompt Injections and Data Exfil
Manus RCE via prompt injection
Prompt Injection in Document triggers agentic workflow
Prompt injection into Grok with hidden ASCII unicode tags
prompt injection in research papers
Indirect Prompt Injection via documents to create Jira tickets via MCP
Examine indirect prompt injection via a document or email, where an agent summarizes text and sends it to a specified address, highlighting automation risks and accountability.
Discover how indirect prompt injection occurs when the Google Calendar connector reads a calendar event description, exposing AI agents to unsolicited invites and injected prompts.
Explore indirect prompt injection via skills files in frontier models, and learn to review downloaded skills, test for ascii unicode prompts, and guard against data exfiltration in penetration testing.
Demonstrate indirect prompt injection using Grok to auto-create a GitHub repository from a document via Krok connectors, highlighting write access and automated actions on GitHub.
Explore the loop library to test agentic loops with iterative feedback and conditional prompting, including a refund loop example that demonstrates prompt injection and red-teaming opportunities.
Explore how indirect prompt injection in documents can generate phishing links and drive-by malware, exposing risks in large language models like Gemini and other AI systems.
Demonstrates a zero-click data exfiltration vulnerability in a chat bot integration, using a markdown image render payload to exfiltrate conversation data from chat interfaces like Rakuten Fiber with GPT 5.5.
Explore how indirect prompt injection defeats Apple email summary and calendar invites, revealing vulnerabilities in AI summarizers and the need for stronger guardrails to prevent sensitive data leaks.
Demonstrates indirect prompt injection that chains actions to exfiltrate PII from G drive via email, highlighting data leakage risks in security testing.
Explore how indirect prompt injection triggers agentic actions to auto-schedule fake meetings via calendar invites and emails, revealing risks through temp-mail.org and calendar connectors.
Explore indirect prompt injection and memory exfiltration by generating an image from memory data and uploading it to a Google Drive folder, highlighting a dangerous automation pipeline.
Prevent sensitive information disclosure by enforcing strict access control and least privilege. Guard PII, financial and HR data with context aware filtering and redaction in RAG storage to avoid leaks.
Explore how sensitive information disclosure occurs in AI systems, including PII and training data, and how encryption, key management, and strict access controls prevent leaks.
Demonstrates a data exfiltration attack in a learning management system using a prompt-injection payload in markdown to leak information when a user clicks a link, revealing markdown rendering vulnerabilities.
Enumerating Grok's container dives into a Linux VM inside a container to inspect processes, memory, and binaries, and demonstrates how outbound network activity and GitHub connectors can enable data exfiltration.
this lecture explains supply chain vulnerabilities in lms, including third-party components, pre-trained models, plugins, and open-source code, with vetting, provenance, zero-trust, and backdoor awareness.
Examine excessive agency in LMS, where AI agents call APIs, invoke tools, or browse the web, risking data disclosure and unintended actions; apply guardrails, least privilege, and human oversight.
Examine insecure plugin design and how missing access control threatens LLMs through untrusted plugins. Learn defenses like code reviews, isolation, and granular authorization to prevent data exfiltration and prompt injections.
Demonstrates exploiting LLM APIs with excessive agency by prompting an AI to access back end APIs and delete a user, revealing how prompt injection bypasses front-end controls.
Explore system prompt leakage in the OWASP top 10, and learn how adversaries encode prompts with base64 or leet speak, exposing hardcoded API keys and bypassing the input filter.
Claude Opus 4.6 Prompt Leak
ChatGPT Prompt Leak
Short prompt to leak system prompt
Ultra short prompt to leak prompt
Claude Memory and Prompt Enumeration
Universal Prompt Leak Payload
Claude Code Prompt Leakage
Cluely - system prompt leakage
Claude Opus 4.6 and 4.7 prompt leaks
Explore how frontier models leak their tools by crafting xml-like prompts (xmi) with a tools root and t tags listing available tools, highlighting connectors and internal tools.
examine misinformation in large language models, including hallucinations, misclassification, bias, and harmful output, and outline mitigations such as fact checking, source verification, confidence scoring, and human oversight.
Demonstrates how llms can hallucinate and spread misinformation due to overreliance. Highlights production risks and shows how lowering the temperature improves predictability for cybersecurity tasks.
Demonstrates how prompt injection can cause LLMs to provide misinformation, highlighting overreliance risks and the need for oversight when deploying internal chatbots.
DeepSeek bias leads to insecure code
Grok 4.1 Jailbreak to generate false documents
Jailbreaking Grok to generate harmful social media images
The Ultimate AI/LLM/ML Penetration Testing Course
Your instructor is Martin Voelk. He is a Cyber Security veteran with 25 years of experience. Martin holds some of the highest certification incl. CISSP, OSCP, OSWP, Portswigger BSCP, CCIE, PCI ISA and PCIP. He works as a consultant for a big tech company and engages in Bug Bounty programs where he found thousands of critical and high vulnerabilities.
This course has a both theory and practical lab sections with a focus on finding and exploiting vulnerabilities in AI and LLM systems and applications. The training is aligned with the OWASP Top 10 LLM as well as the OWASP Top 10 Agentic vulnerability classes. The videos are easy to follow along and replicate.
The course features the following:
· Prompt Injection
· Sensitive Information Disclosure
· Supply Chain
· Data and Model Poisoning
· Improper Output Handling
· Excessive Agency
· System Prompt Leakage
· Vector and Embedding Weaknesses
· Misinformation
· Unbounded Consumption and DoS
· OWASP PwnzzAI Shop
· OWASP Finbot (new)
· OWASP Top 10 for Agentic Applications
· Portswigger - Agentic AI Labs
· Prompt Airlines CTF Challenge Walkthrough
· SecOps Group AI/ML Mock Exams 1 & 2 Walkthrough
· OWASP Finbot CTF (old)
· Selara Jailbreak Game CTF
· Gandalf Agent Breaker CTF
· Hack The Agent CTF
· AI Prompt Attack and Defense Game Tensortrust
· Crowdstrike AI Unlocked Challenge
· Game Arena Challenges
· PromptTrace Prompt Injection Labs
· PromptInjects CTF
· 8ksec CTF
· AIPWN CTF
· Other CTFs
· Jailbreaking
· AI Browsers Attacks
· AI Coding Agents Attacks
· MCP Attacks
· Multimodal Attacks (Images, Audio and Video)
· Tooling
Notes & Disclaimer
Portswigger labs are a public and a free service from Portswigger for anyone to use to sharpen their skills. All you need is to sign up for a free account. I will to respond to questions in a reasonable time frame. Learning Pen Testing / Bug Bounty Hunting is a lengthy process, so please don’t feel frustrated if you don’t find a bug right away. Try to use Google, read Hacker One reports and research each feature in-depth. This course is for educational purposes only. This information is not to be used for malicious exploitation and must only be used on targets you have permission to attack.