Deep Agent - Multi Agent RAG with Gemini and Langchain

Explore how ai agents are built with lm memory tool and planning tools, guided by a pocket dictionary mind map that outlines agent patterns and when to use each pattern.

Differentiate large language models from AI agents, and explore the engine of autonomy, the think and answer loop. Present a four-type agent taxonomy and a theoretical blueprint.

An AI agent autonomously perceives with sensors, reasons in Elm, and acts through tools and memory to solve problems.

Identify four major agent patterns: react agent, plan and execute agent, multi-tool agent, and self-corrective agent. Learn how these patterns plan, act, verify, and adapt for real projects.

Choose the right agent architecture by task characteristics: use react for simple queries, plan and execute tasks for complex ones, multi tool for diverse tools, or self-corrective for reliability.

Master Google Gemini AI Studio and Lang Smith; set up API keys, claim Google Cloud credits, enable agent tracing, generate video with Nano Banana Pro, and test environment variables.

Explore google ai studio to test google gemini three, and generate images with nano banana and videos with vivo 3.1. Save chats to google drive and adjust settings for testing.

Compare Google Gemini models and pricing to choose the right model. Learn about context window, token costs, thinking levels, and grounding with Google search in Python and Langchain.

Generate a Lang Smith API key, configure env variables, and verify loading in the notebook for agent tracing via the Lang Smith dashboard.

Test your Google Gemini API key using the sample code from Google AI Studio, then validate Lang Smith integration and Lang chain setup for multi-agent RAG workflows.

Demystify loop engineering by expanding a react agent into a five-step loop with two models, self-prompting, and memory shared across agents to refine outputs.

uncover gemini 3 and langchain bootcamp basics, compare gemini 3 with gemini 2, and learn to generate infographic outputs by analyzing PDFs and images with multi-modal tools and tool calling.

Configure a Gemini three notebook with internal Google search, add an external web search tool and a weather API, and set up API keys and environment variables for LangChain v1.

Learn LangChain message types and ChatGoogleGenerativeAI to connect the Gemini three model with Google Generative AI, handling environment variables and LLM messages in a multi-agent workflow.

Learn to use Google Gemini 3 and Gemini 2.5 models with Lang chain in Python, compare pricing and token usage, and implement system and user message flow to invoke Gemini.

Learn how AI message structure uses content, text, and content blocks to represent model outputs. Understand tracing, tokens, and how system, human, and tool messages shape responses.

Learn to stream a model's output with LangChain ChatModel, printing chunks in real time to reduce perceived latency, using Gemini two as the underlying model.

learn to analyze local image files by encoding them to base64 with mime types, then pass the data to Gemini models for analysis using Python.

Analyze a pdf with Google Gemini 3 by encoding the pdf in base64 and setting mime type to pdf. Extract key financial highlights from this quarterly report.

Attach tools to the Gemini model for live web search via the llama web search API, using an API key and Python library.

Create a weather tool with a get weather function decorated as a light chain tool, using an API key from the environment to fetch current weather by location.

Create a reusable tools module by converting scripts into a package with __init__.py for use across notebooks. Bind tools to the Gemini model and observe tool calls versus agent execution.

Upload files to the Gemini model context, create a 30-minute limb cache from uploaded file parts, and configure a Gemini cache with a financial-analyst system instruction and display name.

Explore production context caching to optimize costs and speed, using cached conversations, unique context identifiers, and Gemini-based models to compare Q1 and Q2 2024 revenue and generate infographics.

Generate professional ai infographic images for finance presentations using Google's Nano Banana Pro model via LangChain, handling multimodal data with 16 by 9 aspect ratio and 4K resolution.

Explore how agent states evolve through tool calls and Langsmith tracing, with web search, weather, and stock price queries shaping start-to-end node workflows in Langchain and Gemini rag.

Learn how to enable short-term memory by storing chat histories in sqlite, including setting up a sqlite3 connection, a multi-threaded financial agent db, and attaching a checkpoint to persist conversations.

Learn to control total model and tool calls in a LangChain powered agent using model call limit, tool call limit, and model fallback middlewares, with per-thread and per-run configurations.

Use guardrails and PII detection with LangChain to redact emails, URLs, IPs, and API keys before the model or after the tool call, with reusable PII middleware for end-to-end privacy.

Integrate a to-do list planner as the planning layer for a multi-agent system, enabling the agent to generate and execute steps before answering.

Generate structured JSON outputs using a Pydantic base model with keys like company, stock symbol, current price, analysis, and recommendation. Use this schema to guide agent responses for fixed-key data.

Build a multi-agent finance research workflow by linking a stock researcher agent with an MCP agent through a to-do planner middleware and Yahoo Finance MCP server.

Learn to craft a structured system prompt for a finance research assistant, connect Yahoo Finance MCP tools, and use stock data like get stock info and historical prices.

Connect the Yahoo Finance MCP server with a multi-server MCP client, then implement an async get_tools method to fetch and print tool names for the agent.

Connect Yahoo Finance MCP tools to a Gemini-based agent, building async finance research workflows that invoke tools, assemble messages, and return Apple stock data and latest news.

Create a stock researcher agent using a structured system prompt. Leverage available tools like the finance researcher tool and to-do list middleware to analyze stock performance and provide data-driven recommendations.

End-to-end testing of the stock researcher agent and the MCP agent in a multi-agent RAG setup. Demonstrates planning, tool use, and stock market data analysis with tabular comparisons and recommendations.

Explore multimodal rag design for financial documents, focusing on storing and retrieving images within a vector database while balancing accuracy, cost, simplicity, and scalability under real constraints.

Explore five multimodal rag design strategies, focusing on a low-cost, text-only embedding approach with no image inference, converting images and videos to text descriptions using a single collection.

Ingest and convert pdf data into a searchable format, extracting metadata, images, and tables as markdown. Implement a hybrid retrieval system using cosine similarity, bm25, and reranking with cross encoders.

Master data extraction from pdfs using the doc link library to convert images, text, and tables into markdown, with metadata filtering and vector db retrieval.

Plan and implement an automated pdf data extraction pipeline with doc ling, converting pdfs to markdown, images, and tables, and organizing outputs into markdown, images, and tables directories.

Design a production-ready document processing pipeline that stores raw data, markdown, images, and tables by pdf name with page-wise context and enforces images of at least 500x500.

Extract metadata from file names to populate document type, company name, fiscal year and quarter, plus runtime metadata like page number and content type for deduplicated vector db storage.

Learn to convert a PDF to a doc link converter using a PDF pipeline with page and image generation, adjust image scale, and export markdown or tables.

Convert pdf data to markdown with a doc link converter, extracting images and tables, and export with a page break placeholder to preserve page numbers for vector DV.

Identify image items in a document with the doc converter, extract full-page images, and save them alongside page numbers. Process single PDFs first, then scale to multiple files.

Extract and save PDF pages as images by validating size over 500 and assigning page numbers from provenance. Store as PNG using Doc Link and Pillow.

Extract tables from markdown by identifying lines starting with a pipe operator, capture two preceding paragraphs as context, and save tables with page numbers for LM inference and data generation.

Track the table number in markdown illustrates detecting tables in markdown data with the pipe operator, eliminating empty lines, and associating each table with its page context.

Learn to extract markdown tables with context by locating the pipe operator, collecting table lines, capturing the two preceding context lines, and returning content with the next index.

Convert bulk pdf data by packaging into a single extract_pdf_content method, reading all pdfs from a data directory, and automatically producing markdown, images, and tables through a batch conversion.

Verify data integrity by confirming PDF extraction into markdown, tables, and images across multiple companies, ensuring accurate ingestion and preparing image descriptions with a vision model for text data.

Explore generating image descriptions with the Gemini vision model, converting images to markdown, and storing captions in the meta folder to power a multi-modal rag pipeline and bulk processing.

Describe images using Gemini multimodal LLMs, export PDF image data, and build a vector store of descriptions by configuring environment variables and using a structured image prompt for multimodal reasoning.

Load an image as base64 to generate its description. Read, buffer, and encode the image as png base64, then embed it in a prompt for the LLM.

Learn to invoke a language model with structured messages and system prompts to generate image descriptions using Gemini, including a reusable method that processes image paths and outputs markdown data.

Generate image description and save it as a markdown file in an output directory, using the image path metadata (company and document name), creating folders as needed and avoiding duplicates.

Generate and save descriptions for all images in the images directory, converting them to markdown. Track progress and ingest multi-modal data into a vector space for Rag retrieval.