AI Red Teaming: Test, Evaluate & Improve AI Systems

This course contains the use of artificial intelligence.

Course Description

This course provides a practical introduction to AI Red Teaming — the systematic process of testing AI systems, identifying weaknesses, evaluating failures, and documenting findings.

You will learn how to examine AI-generated responses beyond surface-level quality and identify problems that may remain hidden behind confident or polished language.

Throughout the course, you will explore important AI failure modes including hallucinations, factual errors, instruction-following failures, reasoning problems, bias, unsafe responses, and hidden omissions.

You will also learn how professional red teamers design effective test prompts, use adversarial testing techniques, compare AI responses, assess the severity and potential impact of failures, and document findings in a clear and evidence-based red team report.

The course includes practical examples and structured exercises designed to help you develop a systematic approach to AI evaluation and red teaming.

By the end of the course, you will understand how to:

• Design purposeful red team tests rather than rely on random prompting

• Identify obvious and hidden failures in AI-generated responses

• Test instruction following, reasoning, factuality, bias, and safety

• Compare multiple AI responses objectively

• Assess failure severity and potential impact

• Document evidence and create professional red team reports

• Apply a structured red teaming workflow to realistic AI scenarios

No previous AI red teaming experience is required. The course is designed to be accessible to beginners while introducing methods that reflect real AI evaluation and testing workflows.

AI Transparency Notice:
This course was developed and directed by Marina Langer with the assistance of generative AI tools. AI technology was used to support the creation of selected educational materials, visuals, narration, and video content. The course structure, learning objectives, topic selection, instructional direction, review, and final editorial decisions were overseen by the instructor.

The goal is not simply to find mistakes in AI systems, but to understand their behavior through systematic, objective, and evidence-based testing.

Test systematically. Judge objectively. Document with evidence.