Gen AI Security Foundations (CPE)

Explore data and model poisoning across training data, open-source model contamination, embedding and interaction attacks, and remote code execution risks, with defenses like trusted data sources, ML-BOM, red-teaming, and monitoring.

Explore how vector and embedding weaknesses in LLM systems expose data through embedding inversion, RAG poisoning, and cross-tenant leakage, and learn mitigations through strict access control and data redaction.

Explore supply chain vulnerabilities across models, data, libraries, and frameworks in the LLM lifecycle. Learn mitigations like BOMs, audits, hashes, licenses, monitoring, and patching to prevent poisoning and breaches.

Examine direct and indirect prompt injection as a security vulnerability that can bypass safety prompts and leak sensitive data. Learn defenses like input validation, regex checks, and human-in-the-loop.

Learn how large language models can disclose sensitive information like PII, financial and health data, and how to mitigate risks through output validation, data masking, and stakeholder education.

Explore improper output handling, including client-side XSS and CSRF and server-side SSRF and remote code execution, and how validation, sanitization, encoding, and parameterized queries mitigate these risks.

System prompts shape model behavior, but leakage can expose secrets, internal logic, and user roles. This enables privilege escalation and unauthorized access, prompting defenses like least privilege and real-time monitoring.

Identify misinformation in gen ai, including hallucination, bias, and overgeneralization, and its impact on healthcare and legal. Explore mitigations: rag with external databases, fine-tuning, cross-verification, human oversight, and automated fact-checkers.

Explore excessive agency in AI agents, limit permissions and plugins, enforce least privilege, and apply human-in-the-loop and input-output sanitization to prevent data leakage and unauthorized actions.

Identify unbound consumption attacks that exhaust resources through unlimited requests and resource-intensive prompts. Implement defenses with timeouts, throttling, strict input validation, rate limits, and glitch token filtering.

Reflect on what could be hardened in your AI system to make the world a safer place, and share comments and ratings to improve future teaching experience.