AI Security Bootcamp: LLM Hacking Basics

Install and set up Docker to begin the hands-on lab in a secure, containerized environment. Download the Docker CLI from the lab manual, install it, and start the Docker engine.

Spin up lab container by creating a new directory, downloading and running the Docker image, then access the insecure chat box via the terminal URL.

Explore prompt injection vulnerability in large language models, showing how attackers send malicious inputs to control output and distinguishing direct and indirect prompt injection.

Explore sensitive information disclosure in LMS output, and examine how an attacker deceives the chatbot into leaking confidential information.

Engage in a hacking lab exercise that tests chatbot responses to manipulation, revealing the model owner's identity and passwords when disguises imitate friends or operators.

Explore real-world attack scenarios arising from overreliance on AI, including accidental plagiarism by alum, misinformation fed to language models causing fake news, and blindly trusting AI-suggested code that introduces vulnerabilities.

Test LLM reliability in a hacking lab by evaluating a personal assistant's accuracy, verifying responses with trusted sources, and highlighting hardcoded credentials and chatbot hallucinations.

Explore how data poisoning acts as an integrity attack by tampering with training data sources to produce false and biased content from chatbots.

Explore real-world attack scenarios that delay service by flooding LMS with complex requests, degrading service for users and driving up costs, while attackers manipulate API rate limits to disrupt availability.

Explore a hacking lab exercise that demonstrates how encoded inputs and excessive permissions can disrupt a chatbot service, leading to system downtime.

Implement input validation to enforce defined limits, restrict API requests per user or IP, and cap queued actions in LM-driven systems, while continuously monitoring resource utilization for abnormal spikes.