Building a RAG application in Python

Set up a basic python application that connects to an llm via the OpenAI sdk, creates a virtual environment, and builds a structured project with a command line interface.

Configure a config-driven LLM client with OpenAI SDK, define message roles (system, user, assistant), and enable streaming responses via a chat client and from_config.

Set up a Python command line interface for a RAG app using Typer, load config, and a chat client, then run a streaming chat loop with reset and exit.

Pull a model from Ollama and configure the .on file, then start ollama serve to test a local llm; outline the rag workflow with a retriever, vector store, and rewriter.

Connect an LLM chat to a Postgres-backed vector store, configure env and settings in config.py, and prepare Docker Postgres for a RAG Python app.

Set up postgres with the pgvector extension via docker-compose and initialize the database using the provided init.sql to create documents and chunks for vector storage in Python.

Define a swappable vector store contract in Python with a Postgres pgvector backend, via stores/base.py, and plan to add Weaviate later, using data classes StoredChunk and SearchHit.

Implement a Postgres vector store by creating postgres.py, establishing a lazy, single connection, and upserting documents with chunks inside a transactional, rollback-capable process.

Implement the postgres store to satisfy the base.py contract by adding delete_document, wiring up the delete SQL and commit, and stubbing search and close to complete the interface.

Define and wire a vector store by implementing the abstract VectorStore interface and a Postgres backend, then create a factory to instantiate the configured store via pg_dsn.

Learn to upsert a document into a Postgres store from Python, including creating the stores package, a root demo.txt, and verifying data in a Postgres client.

Ingest on-disk documents by splitting into chunks with vectors and feed them to the vector store, while configuring embed model, embed dim, and chunk parameters.

Chunk large documents for a RAG store by splitting text into around 1,000 characters per chunk, paragraph-aware chunks with overlap, using a Python chunking module before embedding.

Convert text chunks into numeric embeddings using OpenAI, validate the code with a Python interpreter, and implement an embedder class to produce vectors for a vector store.

Experiment with the nomic embed text model by loading config, creating an embedder, and embedding sample text to yield 2 vectors of length 768, then prepare for the vector store.

Set up the ingestor for the RAG application, building the input pipeline to hash content for idempotence, chunk text, embed in batches, and upsert chunks to a Postgres vector store.

Wire up the cli for the ingestor by configuring logging and building with config, embedder, and store, then expose ingest and scan commands to process files and the documents dir.

Test the ingestion pipeline end-to-end by fixing typos, preparing documents, running pyrag ingest, populating a Postgres-backed vector store with 75 chunks and embeddings, and preparing for retrieval implementation.

Learn to query the vector store and retrieve related chunks using cosine similarity in Postgres, inspecting a 768-dimensional embedding and deciding between Postgres search now and HNSW later.

Implement search in the Postgres vector store using HNSW when beneficial and cosine distance to rank results. Retrieve source paths, chunk indices, content, and metadata, returning the top matching chunks.

Embed the question and perform a pure dense search in the vector store, ignoring the unused queryText parameter for now, and return a retrieved context of hits.

Test retrieval in a python rag app by running queries in the interpreter, using a vector store and embedder to fetch top hits via dense search.

Configure a default system prompt to make the assistant an expert in mythological creatures and to use vector store documents, avoid inventing facts, and allow file-based overrides.

Wrap retrieved context into the user prompt by implementing toPromptBlock and buildUserMessage, formatting hits with source, chunk, and score, then combining them with the original question for the LLM.

Learn to implement a stateless ask command in a Python RAG app, wiring config, top-k retrieval, and end-to-end vector store querying for one-shot use.

Explore implementing a custom external system prompt by loading from prompts/system-prompt.txt and using the system_prompt_file environment variable to override the default prompt in pyrag with vector store support.

Modify the chat command to use rag by querying the vector store and incorporating retrieved context into the response, with configurable top k and dynamic system prompts.

Implement a terminal spinner in Python to provide visual feedback while the language model processes, using a spinner class with ascii frames and threading, started and stopped around the response.

Improve the spinner by silencing log noise and implementing a terminal clear, by setting httpx and httpCore log levels to warning and clearing the line after thinking.

Set up a watchdog-based file watcher to auto-ingest files into the vector store and move ingested files into the process directory with a timestamp to prevent overwrites.

Add a concise safety check to prevent re-ingesting files dropped into the process directory by the watcher. Implement a two-line helper in ingest.py that compares path relative to root.

On app startup, scan the documents directory to ingest existing files and sync them with the store, skipping directories and logging failures; debouncing is addressed in the next lecture.

Implement a per-path debouncing handler that waits 750 milliseconds after file system events—created, moved, or modified—before ingesting files.

Set up a watcher that uses a watchdog observer and a debounced handler to monitor the documents directory non-recursively, perform initial scan, ensure directories exist, and stop cleanly on interruption.

add the watch command to the cli to monitor ingester activity, set up logging, manage the ingester lifecycle, and ensure the database connection closes after use.

Join a hands-on walk through building a python RAG application, processing documents into chunks, and using hybrid search to boost retrieval in a vector store.

Combine dense vector search with lexical ranking to create a hybrid retrieval for a Python rag app in Postgres, using reciprocal rank fusion and content TSV.

Improve rag retrieval by replacing pure dense search with hybrid retrieval that combines dense vector semantic ranking and lexical content tsv ranking, fused via reciprocal rank fusion in postgres.py.

Extend the vector store contract to support metadata by adding a metadata dict to upsert_document and adding document_metadata and chunks_metadata fields, updating base.py and postgres.py.

Update the postgres vector store to support document metadata in upsert operations by adding a jsonb metadata field, serializing with json.dumps, and wiring it through insert and on conflict paths.

Update the search method to return document metadata and ingested at by selecting documents.metadata as document_metadata and including it in each search hit, without changing semantic, lexical, or fusion logic.

Update ingest logic to populate document and chunk metadata by extending upsertDocument with suffix, kind, and type text, then ingest and verify metadata in Postgres and the vector store.

Add a second vector store with Weaviate alongside Postgres, using the abstract interface to implement Weaviate.py and run docker-compose with the Weaviate profile.

Configure the Weaviate store by defining host, http port, gRPC port, and collection, loading values from the environment with defaults like local host, 8080, 50051, and PyRegChunk.

Set up the Weaviate v8 vector store in Docker Compose, configure the store, and create a lazy-connected collection for a Python rag app.

Define hasDocument to verify the store contains a document by source path and content hash using a lazy connect and a filtered collection query, returning true if totalCount > 0.

Define upsertDocument in the Weave 8 store, delete old chunks, batch-insert new chunks with metadata and vector embedding, and ensure atomicity with error handling.

Implement delete document by removing all chunks for a given source path using deleteMany with a where clause on the source path. Proceed to the upcoming search method.

Implement a Weaviate vector store search with a hybrid query, returning ranked hits and metadata. Add safe JSON parsing and a date-time helper, and ensure proper client closure.

Configure a Weaviate vector store in the rag app by updating the factory to return a Weaviate store with host, http port, grpc port, and collection from config.