Complete RAG Testing course with Ragas Deepeval and Python

Learn how to install the Python pip utility, download get-pip.py, run it to install pip, and use pip install to add dependencies like openai for hands-on demos.

Create a new secret API key from the API keys panel, name it demo, and copy the key for all demos and ChatGPT interactions. Keep it private; do not share.

Explore four popular rag techniques—standard rag, corrective rag, speculative rag, and graph rag—plus knowledge graphs and multi-document k-top scoring.

Clone the rag test repository from GitHub, create a rag test folder, install dependencies from requirements.txt with pip, and prepare the environment (venv or conda) for a simple rag pipeline.

Learn how prompts are tokenized into tokens, converted into vector embeddings, and retrieved from documents by chunking with a defined size and overlap for effective top-k results.

Explore implementing an augmentor in a retrieval-augmented generation workflow by combining a system prompt with the top three retrieved contexts to guide a foundation model.

Explore how to retrieve top k documents or chunks from a knowledge base in a rag system, using a vector database and cosine similarity to fetch the most relevant documents.

Identify the most relevant document, retrieve its top three chunks, and use a two-stage rag approach with metadata and cosine similarity for accurate answers.

Explore how a vector database uses cosine similarity on token embeddings to retrieve top-k document chunks, and test retriever performance with precision and recall.

Test the generation stage by feeding prompts into a llm, assess precision, recall, and truthfulness, and verify contextual accuracy using the ragas library and deep eval.

Explore specific query synthesizers in rag testing, using precise, single-hop retrieval to extract named entities from document chunks, with contrast to abstract queries.

Master multi-hop reasoning for complex, specific queries by navigating linked information chunks from the retriever and validating information across sources.

Learn how to construct the ground truth table as the reference for evaluating entity recognition, then compare predicted tables to compute F1 score and accuracy using a test oracle.

Learn to test context precision with the ragas evaluate library, building a ground-truth dataset and measuring answer relevancy to achieve a context precision score.

Demonstrates context recall in a python demo using ragas, showing reference, context, and answer, with a 0.5 threshold and a context recall average of 0.857.

Explore RAG metrics like context relevance, precision, recall, and relevancy, and learn how this library uses generated questions and cosine similarity of embeddings judged by an LLM.

Evaluate rag pipelines using fluency, coherence, relevance, and concision, with an LLM as judge and deep eval benchmarks for reliable evaluation.