Getting started with Gen AI using LlamaIndex for Beginners

Stay current with Llama Index through blogs and docs, and connect via the Discord community and newsletters. Explore the knowledge base and documentation with AI-assisted answers and code examples.

Explore OpenAI models and how llama index defaults to GPT-3.5 turbo for text generation and text embedding Ida 002 for retrieval and embedding.

Learn how to set up your OpenAI API key, create and store the secret key, and manage prepaid billing to run models like GPT-4 and 3.5 turbo.

Explore multiple methods to set up your OpenAI API key, including using Windows environment variables, a .env file, loading with Python dotenv, and passing the key during model initialization.

Install Lama index with pip, download the required packages and subpackages, and upgrade or check the installed version using pip show Lama index.

Learn how llama index builds a document object as the base for index applications, and splits it into nodes, manually or with loaders like the simple directory reader.

Learn how the SimpleDirectoryReader loads diverse file types into a document object. Explore how VectorStoreIndex converts those documents into vector embeddings for in-memory storage or vector databases.

Load files from a directory (and subdirectories) into document objects with simple datareader from lambda index, then create a vector store index for querying.

Load multiple text files with a simple data reader, create documents, and build a llama index vector store. Query across the files to answer questions with a Rag app.

Learn to read and query CSV files with Lama index using simple directory reader and CSV reader, build a vector store index, and run queries on employee data.

Learn to query PDF files with Lama Index by converting two PDFs into documents and querying them with a query engine. Discover pre-training costs and federation versus sharding in practice.

Learn to process Excel files with Lama index via the Lama Hub and Pandas Excel reader, loading data and querying the country with the most records and the oldest age.

Learn to process doc files with Lama index, convert Nvidia and Tesla articles into documents, build indexes, and query the engine to extract stock surge drivers and industry challenges.

Discover how llama parse and llama pass transform complex documents into structured data, integrate with llama index, and support pdfs, presentations, images, and spreadsheets.

Learn to use llama parse to query complex PDFs by loading environment variables and llama pass API keys, and choose text or markdown outputs for parsed results.

Learn how to persist Gen AI indexes to disk using a storage context, store and load from storage, and query without reloading data into memory.

Explore how LlamaIndex offers vector store, list, summary, tree, and keyword table indexes to perform similarity search, summarization, and keyword-based retrieval across document nodes.

Learn how to perform insert, delete, update, and refresh operations on indexes in llama index, including creating document objects, assigning doc IDs, and refreshing the index to apply changes.

Refresh and persist LamaIndex indexes across folders using a recursive directory reader and file name as id to handle added or changed documents in storage.

Customize Lama Index by adjusting the default underlying LM model, chunking documents into smaller chunks, setting the number of output tokens, and tweaking additional parameters to fit your needs.

Configure the LM model with llama index’s openai package, initialize it, and set a model with settings using GPT 3.5 turbo or GPT four mini, with temperature and max tokens.

Learn how to customize chunk size and chunk overlap when ingesting documents into an index. Balance performance and fidelity with practical code examples using global settings or sentence splitters.

Explore all possible customization in llama index, from basic setup to advanced options like chunking, persistence, external vector stores, various models, streaming responses, and chat mode.

Enable streaming response to reduce latency by printing results as they are generated, using two steps: ensure a supported LMS and configure streaming to two while wiring the query engine.

Enable streaming response by defining a custom model with streaming true, enable streaming in the query engine, and print the streaming response using print response string.

Explore the chat engine’s high level flow, comparing condense and react modes. Condense mode queries the knowledge base by default; react mode uses an agent to decide queries.

Explore react mode in the LlamaIndex chat engine, enabling a LangChain chat OpenAI custom model and using the query engine tool to observe and answer.

Learn to build a condensed mode chat engine with LlamaIndex by initializing a storage index, loading documents, and using a chat workflow for interactive Q&A with verbose debugging.

Explore how prompts shape LlamaIndex applications, from default and custom prompts to chat and non-chat prompts, enabling tone control, reference only to the knowledge base, and reduced hallucination.

Apply hands-on customization of prompts with a custom template and refine template using llama index, combining context with prior knowledge to shape responses.

Expose your existing Python code as an API using Flask by setting up a basic server, creating endpoints, and handling queries to return document-based responses.

Expose a Flask API to query a locally stored vector index; initialize or load the index and persist index files, using a config.ini to securely manage the OpenAI API key.

Expose a Flask endpoint to query an initialized index using a text parameter, handle 400 and 200 responses, and test the endpoint with Postman for a reusable backend.

Expose your index app as a FastAPI endpoint, and learn Pydantic models, data contracts, and validation rules, including strict vs flexible typing and uvicorn serving.

Expose your app as a FastAPI endpoint, booting at startup to load data and build a persisted index, then load the index to answer queries via a post endpoint.

Explore how vector databases store high-dimensional embeddings of unstructured data and enable fast vector similarity search for semantic search, recommendations, and anomaly detection.

Learn to build LLM apps with ChromaDB by initializing a ChromaDB client, creating a collection, and using a ChromaDB vector store to ingest data, embed documents, and perform queries.

Discover how to use MongoDB Atlas as a cloud storage for embeddings and indexes, connect via a connection string, and create a vector search index for llama index.

Connect to MongoDB using document store and index store in lambda index storage, test the connection, and store docs and indexes in Atlas for retrieval with vector storage comparison.