Building a RAG Application From the Ground Up

RAG provides a way to incorporate user data into an AI query by processing the data and storing it in a numeric vector "database".  In the demo project, our raw data is a set of four PDF files.  We must convert the PDF's to a text format.  This video shows you how to convert PDF's to Markdown text.  This is the first step in a pipeline that will ultimately store the data in a FAISS index.

After converting user data to a text format in Section 2, we are now ready to "ingest" the data into a FAISS index.  We're using the ai/embeddinggemma model to map out the data in a multidimensional space.  This layout arranges data so that searching for content that matches AI Query input can execute quickly.  The FAISS Index is made up of two files: index.faiss and index.pkl.  Together these files are responsible for storing and retrieving data with semantic similarity.

This video demonstrates how to execute the ingestion script as well as a test script to verify that the dimensions of the vectors are calculated to 768 as expected.  The ingestion script reads in the user data markdown documents and with the use of chunking and embedding creates the vector store that stores data in two files that make up the FAISS index.  This video will help to understand how data is stored in numeric vectors (index.faiss) and then human readable content can be extracted using meta data stored in a "pickle" file (index.pkl).

The code in app.py is broken down into nine sections as shown below.  The application implements that query pipeline from User question to user data extraction and ranking for similarity to the question.  A PROMPT is created that will hold system instructions (guardrails), context and the user question.  Context will be provided by data from the FAISS index.

• Global Configuration

• Section 1: Environment and Docker Model Runner Endpoint

• Section 2: Load Embedding Model

• Section 3: FAISS Vector Store (only use indexes you create yourself)

• Section 4: Large Language Model

• Section 5: Create Retriever

• Section 6: Cross Encoder ReRanker

• Section 7: RAG Prompt

• Section 8: RAG Chain

• Section 9: User Interface

This video will walk you through running the app.py script.  The script provides provides instruction to the user who  wants a question answered.   The user question is added to the application PROMPT, along with static system instructions, and context.  Context is provided by extracting text data from the FAISS index and comparing it to data in the users questions.  The similarity comparisons are scored and the top 4 data chunks are merged and provide the context.  The video explains using the huggingface.ai website to set up an access token that is tied to the CrossEncoder library which scores the comparisons between user data in the FAISS index and the text in the user question.

The DEBUG demo outputs data related to document data in the vector store and total chunks in the index and chunks per source.  Once the user asks a questions the ReRanker is called to generated top 4 similarity matches to the user questions.  This outputs data showing ReRanker similarity scores for documents matched to the user question.

If you are reading the code, and don't fully understand what the code is doing, I recommend using AI to help you to understand it.  This is a short video that can help you to prompt your favorite AI to explain what the instructions in the code are doing.

We conclude this course with a discussion of memory as it pertains to 'how much data can we add to the FAISS index' and how much RAM is needed to run this application locally with Docker Model Runner.  We also address the question: 'Should I run locally or should I set up my RAG application in the cloud?'.  We look at a list of use cases for running RAG in the cloud.  The "Next Step" is to keep learning and there are suggestions for creating Agentic RAG.