Name: How to test or Evaluate Gen AI, LLM, RAG, Agentic AI
Rating: 4.1 (48 reviews)

Udemy Business

Teach on Udemy

Turn what you know into an opportunity and reach millions around the world.

Learn More

Your cart is empty.

Keep shopping

Created bySoumen Kumar Mondal

Last updated 8/2025

English

What you'll learn

Learn the end-to-end process of evaluating LLM applications. Quality criteria to choosing the right evaluation method, metrics for RAG, Agentic AI.
Learn Evaluation of RAG with RAGAs framework. Understand RAG and what are the component to evaluate of RAG.
Learn how to evaluate RAG with context precision, recall metrics. How to test RAG application with Python and RAGAs.
Learn how to test and evaluate RAG application with Pytest. Learn API automation of RAG application.
Learn how to test and evaluate Agentic AI application using DeepEval. Automation testing of Agentic AI using Pytest.
Learn how to trace RAG application using LangSmith. Create evaluation dataset using Python. Evaluate with Dataset using LangSmith.

Course content

6 sections • 17 lectures • 2h 47m total length

Overview of the course4:07

RAGAs Framework overview5:11
In this lecture, you’ll explore the RAGAs (Retrieval-Augmented Generation Assessment) framework—a structured, community-driven approach to evaluating RAG pipelines. You’ll learn how RAGAs breaks down RAG systems into measurable components like retrieval precision, faithfulness.
Code for following Ragas and RAG app0:15
Context Precision Metric with Ragas - HandsOn15:40
In this session, you'll dive into one of the key evaluation metrics in the RAGAs framework—Context Precision. You'll learn how this metric measures the proportion of retrieved context that is actually relevant to the question being asked. We'll walk through a hands-on implementation using a sample RAG pipeline and evaluate it using RAGAs.
Context Recall , Faithfulness metric of Ragas - Handson10:37
This hands-on session focuses on two crucial metrics from the RAGAs framework—Context Recall and Faithfulness. These metrics help assess the quality of your RAG application's retrieval and generation stages:
Context Recall measures how well the retrieved context covers all the information required to answer the question.
Faithfulness checks if the generated answer is grounded in the retrieved context or if it hallucinates.
Evaluate RAG application - Handson17:33
In this hands-on session, you'll learn how to evaluate a complete Retrieval-Augmented Generation (RAG) pipeline using the RAGAs framework.
Understand RAG API application - Handson (Will use for API testing)9:33
In this session, you'll explore how a RAG-based application is exposed as an API, and how to evaluate its behavior through automated API testing. You’ll learn how to design inputs, interpret JSON responses, and connect evaluation tools like Pytest or custom scripts to validate retrieval quality and generated answers.
Evaluate RAG application with Pytest API testing approach18:37
In this session, you'll explore how a RAG-based application is exposed as an API, and how to evaluate its behavior through automated API testing. You’ll learn how to design inputs, interpret JSON responses, and connect evaluation tools like Pytest or custom scripts to validate retrieval quality and generated answers.

Requirements

Very basic idea of Python and Generative AI.

Description

Evaluating Large Language Model (LLM) applications is critical to ensuring reliability, accuracy, and user trust—especially as these systems are integrated into real-world solutions. This hands-on course guides you through the complete evaluation lifecycle of LLM-based applications, with a special focus on Retrieval-Augmented Generation (RAG) and Agentic AI workflows.

You'll begin by understanding the core evaluation process, exploring how to measure quality across different stages of a RAG pipeline. Dive deep into RAGAs—the community-driven evaluation framework—and learn to compute key metrics like context relevancy, faithfulness, and hallucination rate using open-source tools.

Through practical labs, you'll create and automate tests with Pytest, evaluate multi-agent systems, and implement tests using DeepEval. You'll also trace and debug your LLM workflows with LangSmith, gaining visibility into each component of your RAG or Agentic AI system.

By the end of the course, you’ll know how to create custom evaluation datasets and validate LLM outputs against ground truth responses. Whether you're a developer, quality engineer, or AI enthusiast, this course will equip you with the practical tools and techniques needed to build trustworthy, production-ready LLM applications.

No prior experience in evaluation frameworks is required—just basic Python knowledge and a curiosity to explore.

Enroll and learn how to evaluate or test Gen AI application.

Who this course is for:

Beginners, Testers, Quality Engineers, SDET, Data Engineers, Data Scientists, Evaluator, Gen AI developer, Students, Researchers, Project managers

What you'll learn

Explore related topics

Course content

Introduction1 lecture • 4min

Theory - Evaluation process and RAG2 lectures • 13min

RAG Evaluation with RAGAs Framework7 lectures • 1hr 17min

DeepEval3 lectures • 41min

Langsmith4 lectures • 32min

Quizzes0

Requirements

Description

Who this course is for: