【한글자막】 랭체인 - LangChain 으로 LLM 기반 애플리케이션 개발하기

In this video, we will be focusing on building an application that connects with LinkedIn and utilizes LLMs to analyze data that we scrape from Linked.

Here are the topics that we will cover:

1. Scraping LinkedIn API Data:
   We will write a code that fetches and scrapes information from a LinkedIn URL using the function called "scrape_linkedIn_profile".

2. Using ProxyURL:
   We will learn how to use ProxyURL to scrape LinkedIn API data. We will see how it's advantageous to use this tool.

3. Cleaning Response Payload:
   We will learn how to clean unnecessary data from the payload. We will discuss the importance of being frugal with tokens to avoid exceeding the token limit and how it affects the API call made to LLMs.

4. Leveraging summarization  Power of LLMs:
   We will give the LinkedIn data that we got as an input to an LLM chain so it can output a coherent summary.

LangChain Agent Theory

In this video, we’ll dive into the concept of LangChain agents and how they can be used for reasoning and performing tasks and interact with external services.

Topics:

• Introduction to LangChain agents

• The concept of subtasks and how agents split them to complete bigger tasks

• Chain Of Thought Prompting

• ReAct Prompt Prompting

• How agents can be used to perform tasks and interact with external services

• Understanding the structure of agents and their tools, Toolkits

• A practical example of building an agent

Summary:

• Introduction: The video outlines the necessary imports and main class initializations for the ingestion phase of a RAG (Retrieval-Augmented Generation) pipeline.

• Environment Setup: It covers setting up environment variables for API keys (OpenAI, Pinecone, LangChain, Tavily) and configuring LangSmith for tracing. It also explains the setup of SSL context using certifi to handle HTTP requests securely, with a note on disabling VPNs on corporate networks to avoid certificate errors.

• Logging: A pre-prepared logger.py file is introduced to provide colored and readable logs for tracking the ingestion process, with functions like log_info, log_success, and log_error.

• Core Imports: The video details the essential libraries and classes to be imported:

  • LangChain Text Splitters: RecursiveCharacterTextSplitter is imported for breaking down large documents into smaller chunks.

  • Vector Stores: It shows how to import and use either a local vector store like Chroma or a cloud-based one like PineconeVectorStore. The presenter opts for Pinecone for the demonstration.

  • Embeddings: OpenAIEmbeddings is used to convert text documents into vector representations. It also mentions that open-source alternatives can be used.

  • Data Loading: TavilyCrawl, TavilyExtract, and TavilyMap are imported for web scraping and data extraction.

  • LangChain Core: The Document class is imported, which is the fundamental data structure in LangChain for handling text with associated metadata.

• Class Initialization: The video walks through initializing the main classes:

  • OpenAIEmbeddings: Initialized with the text-embedding-3-small model. It explains how to manage rate limiting using chunk_size and retry_min_seconds to avoid API errors when processing large batches of documents.

  • PineconeVectorStore: The presenter demonstrates creating a new index in the Pinecone UI and then initializes the PineconeVectorStore class in the code, linking it to the newly created index and the embedding function.

  • Tavily Tools: Initializes TavilyCrawl, TavilyExtract, and TavilyMap for use in the pipeline.

• Sanity Check: The video concludes by running the script to ensure all imports and initializations are working correctly without any errors.

Summary:

• The video shows how to use the TavilyCrawl tool to automatically crawl and extract documentation from the LangChain website.

• Crawling is an automated process of navigating a website by following hyperlinks to gather related content, which is a vital capability for AI agents.

• The demonstration begins with setting up boilerplate code using asyncio to run the main crawling function.

• The video then details how to use the TavilyCrawl tool by invoking it with the URL of the LangChain documentation (https://python.langchain.com/).

• It explains and demonstrates how to use the max_depth parameter to control how deep the crawler explores from the base URL and the extract_depth parameter to retrieve more detailed content like tables and embedded media.

• A key feature highlighted is the ability to provide natural language instructions to the crawler, which allows for more precise and relevant content extraction by filtering pages based on specific topics.

• The results of the crawling process are shown in debug mode, detailing the structure of the output which includes the URL and raw content of each crawled page.

• Finally, the video explains how to convert the crawled results into LangChain document objects, including metadata such as the source URL, in preparation for the next steps of the RAG pipeline like splitting and embedding.

Summary: Extracting LangChain Documentation at Scale

This video demonstrates a robust, scalable pipeline for extracting the entire LangChain documentation. The process uses the Tavily library for sitemap discovery and content extraction, combined with Python's asyncio for high-speed, concurrent processing.

Key Concepts & Steps:

• Objective: To map and scrape all pages of the LangChain documentation (python.langchain.com) and convert them into structured LangChain Document objects.

• Built for Scale: The solution is designed to be highly efficient by processing hundreds of URLs concurrently, reducing a task that could take hours to just a few minutes.

Step 1: Website Discovery (Mapping URLs)

• The pipeline begins with a discovery phase using a tool called TavilyMap. This tool is pointed at the main documentation URL.

• It then automatically explores the website's sitemap to generate a complete list of all individual documentation pages.

• In the video's example, this step successfully identifies over 500 unique URLs to be processed.

Step 2: URL Batching for Efficient API Calls

• To process the URLs efficiently and avoid overwhelming the server with hundreds of individual requests, the pipeline employs a batching strategy.

• The full list of over 500 URLs is broken down into smaller, manageable chunks (e.g., batches of 20 URLs each).

• This is a critical optimization because the content extraction API (TavilyExtract) is designed to handle multiple URLs in a single call, which is much faster than processing them one by one.

Step 3: Concurrent Content Extraction

• This is the core of the high-speed pipeline, where two levels of parallelism are used.

• First, each API call already processes a batch of URLs. Second, all of these batch-processing tasks are executed concurrently using Python's asyncio library.

• Instead of waiting for one batch of URLs to finish before starting the next, the program sends out requests for multiple batches at the same time. This asynchronous approach ensures that the program is always busy fetching data, dramatically accelerating the overall extraction process.

Step 4: Structuring Data and Handling Results

• As the concurrent tasks complete, the raw HTML content from each page is processed and structured. The system is designed to be resilient, capable of handling any failed requests or exceptions without halting the entire pipeline.

• For each successfully scraped page, the raw text is formatted into a standard LangChain Document object.

• This object contains the main page_content and, critically, metadata that includes the original source URL. Storing the source URL is a best practice for traceability and citation in RAG systems.

• The final output is a clean, flattened list of these Document objects, ready for the next stage of the RAG pipeline.

Summary:

• RAG Pipeline Overview: The video focuses on the next steps in the RAG (Retrieval-Augmented Generation) pipeline: chunking documents, embedding the chunks into vectors, and storing them in a vector store.

• Chunking (Splitting) Phase:

  • Tool: The RecursiveCharacterTextSplitter from LangChain is used to break down the large, crawled documents into smaller, semantically coherent chunks.

  • Configuration: The splitter is configured with a chunk_size of 4000 characters and a chunk_overlap of 200 characters to maintain context between chunks.

  • Why RAG is Still Relevant: The presenter discusses that despite large context windows in modern LLMs (like 1M or 2M tokens), RAG is not obsolete. It remains crucial for:

    • Cost-Efficiency: Retrieving small, relevant chunks is cheaper and faster than feeding massive documents to an LLM.

    • Precision and Noise Reduction: RAG filters out irrelevant information, reducing hallucinations and overcoming the "lost in the middle" problem of long contexts.

    • Traceability: It allows users to see the source of the information, which builds trust, especially in regulated fields.

• Vector Storage Phase:

  • Asynchronous Indexing: To optimize performance, a new asynchronous function index_documents_async is created to handle embedding and indexing concurrently.

  • Batching: The split documents are divided into batches (e.g., 500 documents per batch) to manage API rate limits from both the embedding model (e.g., OpenAI) and the vector store (e.g., Pinecone).

  • Concurrent Processing: The asyncio.gather function is used to run the indexing for all batches in parallel, significantly speeding up the overall ingestion process.

  • Error Handling: The add_batch coroutine includes a try-except block to catch and log any errors that occur during the indexing of a specific batch, ensuring the pipeline can report failures without crashing.

• Final Logging and Execution:

  • After the indexing is complete, the script logs a summary of the entire ingestion pipeline, including the number of URLs mapped, documents extracted, and chunks created and stored.

  • The video concludes by running the complete ingestion script and showing the process in the terminal, including the final successful completion message.

What is Language Modeling?

• Language modeling is the task of predicting the next word in a sentence.

• It is similar to autocomplete or word suggestions we see in our day-to-day life.

• The language model predicts the probability of the next word based on the previous words in the sentence.

  
  

Formal Definition of Language Modeling

• Language modeling involves computing the probability distribution of the next word in a sequence of words.

• The probability of the next word (x t+1) is calculated based on the sequence of words before it (X1, X2, ..., XT) and needs to be a part of the vocabulary (V).

  
  

Large Language Models: A Brief Overview

• A large language model (LLM) is a language model trained on a huge amount of data.

• LLMs are capable of predicting the probability of the next word with high accuracy.

• They have gained immense popularity in recent times due to their ability to perform a variety of language-related tasks.

How Large Language Models Work

• LLMs work by taking an input of words and predicting the probability of the next word.

• They make their predictions based on the input provided and the probabilities learned during the training phase.

• LLMs can sometimes generate output that is far-fetched from reality and simply not true due to the limitations of probability-based predictions.
  

What is a Prompt in AI Language Models?

• A prompt is the input given to an AI model to produce an output.

• It guides the model to understand the context and generate a meaningful response.

Components of a Prompt:

Instruction

• The heart of the prompt that tells the AI model what task it needs to perform.

• It sets the stage for the model's response, whether it's text summary, translation, or classification.

Context

• Additional information that helps the AI model understand the task and generate more accurate responses.

• For some tasks, context may not be necessary, but for others, it can significantly improve the model's performance.

Input Data

• The information that the AI model will process to complete the task set in the prompt.

• It could be a piece of text, image, or anything relevant to the task.

Output Indicator

• Signals the AI model that we expect a response.

• Sometimes implicit in the instruction, but sometimes explicitly stated.

Here are the key points we'll cover:

• Large language models and their immense knowledge base

• What is zero shot prompting?

• An example of a zero shot prompt

• Why zero shot prompts are popular among AI beginners

• The limitations of zero shot prompting

With zero shot prompting, AI models can generate outputs for tasks they haven't been explicitly trained on, using their pre-existing knowledge to perform the task based on the information provided in the prompt. However, this kind of prompt comes with its own set of limitations, such as accuracy and scope.

In this video, we will explore the concept of Few Shot Prompting, a technique used in prompt engineering that allows AI models to generate or classify new data by presenting them with a small number of examples or shots of a particular task or concept along with a prompt or instruction. Here are the main points we will cover:

What is Few Shot Prompting?

Few Shot Prompting is a prompt engineering technique that involves presenting the AI model with a small number of examples or shots of a task or concept to generate or classify new data that is similar to the examples provided. It is particularly useful in scenarios where there is limited data available for a given task or domain where data may be scarce.

How Does Few Shot Prompting Work?

Few Shot Prompting works by providing the AI model with a few examples of a particular task or concept and a prompt or instruction on how to generate or classify new data similar to the examples provided. It can quickly adapt models to new tasks and domains by fine-tuning existing models without requiring a large amount of new data.

Case Study: Zero Shot, One Shot, and Few Shot Prompting in Action

We will demonstrate the effectiveness of zero shot, one shot, and few shot prompting techniques in generating text-to-text descriptions for Blue Willow, an open source AI tool that generates images from text prompts. By comparing the outputs generated by each technique, we will see which one performed better according to our task of generating a good description to paint a picture.

Introduction to Chain of Thought

• Explanation of Chain of Thought's purpose in improving LLM reasoning abilities

• How Chain of Thought allows models to decompose complex problems into manageable steps

Standard Prompting Limitations

• Examples of insufficient answers with standard zero-shot prompting

• Explanation of zero-shot prompting

Chain of Thought Prompting

• Explanation of Chain of Thought as a new prompting technique

• Examples of Chain of Thought's success in solving complex reasoning problems

• Comparison to human problem-solving methods

Zero-Shot and Few-Shot Chain of Thought Prompting

• Explanation of zero-shot Chain of Thought prompting

• Explanation of few-shot Chain of Thought prompting

• Benefits and limitations of each method

In this video, we will explore the ReAct Prompting technique, a powerful approach to prompt engineering that combines reasoning and acting to accomplish complex tasks. Here are the main points we will cover:

What is ReAct Prompting?

• ReAct Prompting is a technique that allows language models to reason and act upon a task to generate an output.

• It is based on the chain of thoughts that the model can generate to accomplish a task.

How Does ReAct Prompting Work?

• ReAct Prompting involves breaking down a task into multiple steps, reasoning the steps, acting upon them, and then completing the entire task.

• The model can derive an action by accessing external sources or APIs, allowing it to accomplish more complex tasks.

Case Study: ReAct Prompting in Action

• We will look at a research paper that demonstrates the power of ReAct Prompting in action.

• The paper shows how a language model was able to derive the correct answer to a complex question by reasoning and acting upon it.

How to Produce Prompts for Better LLM Responses

Incorporating Context for Coherent and Accurate Responses

• Context provides relevance for generating better responses

• Leaving context to the AI model could lead to off-topic or irrelevant responses

  
  

Defining a Clear Task for Unambiguous Results

• Clear and precise task definitions provide better results

• Ambiguous tasks can lead to confusion and lower performance

• Example: Improving user experience of an e-commerce website

Iterating for Optimized Prompts

• Iterations involve refining prompts and evaluating output

• Refining prompts over iterations leads to optimized prompts

• Taking the time to write effective prompts saves time in the long run

By following these low-hanging fruit tips for prompt engineering, you can improve the performance of your AI models and get better responses. Incorporating context, defining clear tasks, and iterating to refine prompts will lead to more accurate and relevant responses, saving you time and improving the overall performance of your AI models.