OSCP for AI: The GenAI Security Sandbox

Explore direct prompt injection defense by implementing the sandwich defense that splits system and user instructions, translating untrusted input into French and reminding the model not to trust user data.

Expose insecure output handling in llm apps, showing how shell-based command execution enables remote code execution and prompt injection; implement guardrails, remove shell, and validate against a whitelist.

The video demonstrates how insecure output handling in an LLM enables a reverse shell through a command chaining attack, highlighting a blind execution vulnerability.

Explore insecure plugin design by exploiting a vulnerable email system that blindly executes a send_email trigger from an LLM output, lacking validation and enabling internal spear phishing and prompt-injection attempts.

Explore defense against insecure plugin design with a strict validation layer, domain whitelist and human-in-the-loop approval to prevent data leakage and the confused deputy problem.

Explore how LLM-generated cross-site scripting arises when user prompts generate HTML that is rendered unsafely, and learn safe rendering and sanitization techniques to prevent XSS.

Explore cross-site scripting vulnerabilities by using streamlit components to demonstrate how unfiltered ai-generated html or javascript can execute in an iframe, enabling reflected and zero-click XSS and defacement risks.

Explore how a denial of service targets a large language model, exhausting cpu, gpu, and memory through unbounded generation. Enforce max_tokens and timeouts to protect resources.

Implement a hard 50-token limit on the LLM's outputs to act as a circuit breaker, truncating responses and defending against denial-of-service and resource-exhaustion attacks while keeping API costs predictable.

Explore the dangers of insecure history management in llm apps, where session state stores chat history and prompt injection taints context, highlighting sanitation and context-window defenses.

Replace the infinite while loop with a for loop and a max loop limit of five iterations to enforce bounded agency, ensure predictable resources, and trigger human control on termination.

Learn how attackers exploit a vulnerable API with reach return to clone proprietary models, and how to harden the API to return only a final label and apply rate limiting.

Investigate model theft and extraction vulnerabilities where an API reveals full probability scores for public, internal, and confidential categories. Emphasize returning only the final label to prevent leakage.

Explore supply chain attacks in AI development, focusing on the pickle bomb vulnerability in downloaded model weights from HuggingFace, the risk of remote code execution, and safe tensors.

Explore privacy and PII leakage in a language model trained on unredacted data, highlighting lack of filters and guardrails and the need for RBAC and data sanitation.

Protect privacy by adding a PII scrubbing layer at the output stage, using DLP filters, regex and NER to redact credit cards, SSNs, addresses, and medical history.

Examine how audio prompt injection exploits transcription models in a multi-model app, revealing an attack surface expansion when audio is not sanitized, and explore defenses against audio-based prompt hijacking.

Install garak, the Generative AI red team and assessment kit, an open source vulnerability scanner for large language models, using mini conda to create a Python 3.10 environment and llama.

Explore Garak's command-line basics, learn how to view syntax, models, and reporting options; inspect probes, detectors, and buffs; review config, and try interactive mode to test prompts in real time.

Test how Garak interacts with llama2-uncensored to probe jailbreaks like DAN, exposing safety bypasses. Evaluate attack success within the GenAI security sandbox.

Create and activate a conda environment named gizguard_lab with Python 3.10, install GizGuard via pip, then install pandas, langchain, and langchain-community, and verify the installation with version 2.18.0.

Configure Giskard with llama and manage LangChain version differences. Set up an OpenAI compatible API, instantiate a gamma judge with a llama victim and seed prompts for security scanning.

Explains the master scanner workflow for auditing an llm, including fast or deep scans, llama3 model configuration, a timestamped html report, and an interactive menu with 1–7 vulnerability mappings.

Compare jailbreaking, indirect prompt injection, and context leak between mistral and llama3 using a master scanner and OSCP reports; conclude that llama3 performs better overall with no disclosed issues.

Examine hallucination and misinformation in AI using pyrite, detailing how LLMs generate false text and how reports reveal spread of misinformation across five encoding methods, addressing two questions.

Learn how to extend the PyRIT master by adding more prompts across categories like jailbreak and injection, improving the Python script's security audit with comma-separated entries and multiple encoding variants.

Add Gwen2 as a new LLM to the pyrite master script by updating the models dictionary and installing locally, then verify the sha-256 digest.

Adopt an adversarial operator role to tackle five CTF challenges in the OSCP for AI security lab, exploring LLM vulnerabilities through the chat interface and earning quiz codes.