Ethical Hacking against and with AI/LLM/ML (Lite Version!)

Explore what AI and language models are, including universal and custom models, their data-driven training, tokenization, and diverse applications from chatbots to content creation, with ethical and security considerations.

Explore ai/llm threats, including misalignment, bias, and hallucinations, plus backdoors, jailbreaks, prompt injections, plugin risks, and data exfiltration.

Explore common AI attack frameworks and writeups, including lm-attacks.org and MITRE attack resources, plus prompt injection primers, to understand vulnerabilities and defense strategies for AI systems.

Explore prompt injection in AI language models, where untrusted input can cause data exposure and unintended actions, and learn practical labs and prompts for ethical testing.

Explore prompt injection with the Gandalf lab, testing input, system, and output guards across levels. Observe prompts and defenses using burp to understand exfiltration risks.

Examine prompt injection in multimedia content such as video and audio, demonstrated via a YouTube video and Google AI Studio to bypass guardrails in LLMs.

Demonstrate prompt injection using ANSI escape codes to alter terminal output, including colors and cursor behavior, and reveal risks like clipboard exfiltration and potential remote code execution.

This lecture demonstrates using one AI language model to jailbreak another by crafting attacker prompts and crowding strategies, illustrating a proof-of-concept of model-to-model jailbreak.

Explore indirect prompt injection risks in coding assistants like GitHub Copilot and Cursor through instruction files, with a concrete attack that injects deceptive prompts and sign-in links.

Demonstrates how image prompt injection can trigger tool invocation through an MCP server, exposing ethical hacking risks in agentic AI and automated tool calls.

This lecture explains insecure output handling in AI/LLM systems, showing how un-sanitized output can enable cross-site scripting, CSRF/SSRF, privilege escalation, and remote code execution via back-end calls and prompt injection.

Explore insecure output handling in an LMS through indirect prompt injection and cross-site scripting in a hands-on lab, illustrating attacker workflows and defensive lessons.

Expose how training data poisoning compromises LM training data integrity by manipulating sources like Wikipedia and open web text, leading to bias and misinformation in outputs.

Explore denial of service in language models by examining resource-intensive prompts, long queries, and looping inputs that degrade service availability and inflate costs, with emphasis on rate limiting and protections.

Explore supply chain attacks in LMS security, highlighting vulnerabilities from third-party libraries and repos, data breaches, malware, and biases, and promoting zero trust and local sanitization.

Explore permission issues in AI/LLM/ML systems and plugins, focusing on indirect prompt injection and access control to prevent data exfiltration and unauthorized commands.

Examine indirect prompt injection attacks in a lab setting by testing unauthenticated versus authenticated LM access, mapping API surfaces, and triggering hidden prompts to perform actions like account deletion.