Agentic AI & LLM: Build AI Agents with ChatGPT, Ollama & RAG

Tokens are the basic units of text that AI models process and understand.
Instead of reading full sentences like humans, generative AI models break text into smaller pieces called tokens, which can be words, parts of words, or even punctuation.

For example, a sentence like “AI is powerful” may be split into tokens such as “AI”, “is”, and “powerful”. In some cases, longer or complex words may be broken into smaller sub-word tokens.

Tokens are important because they directly impact how AI models work:

• They determine how much text a model can process at once (context window)

• They affect response quality and coherence

• They are used to calculate usage and cost in many AI APIs

Understanding tokens helps you write better prompts, manage input size, and optimize performance when working with large language models.

Embeddings are numerical representations of text that capture meaning and context.
In generative AI, words, sentences, or even entire documents are converted into vectors (lists of numbers) so that machines can understand relationships between them.

Instead of just matching keywords, embeddings allow AI systems to understand semantic meaning. For example, the words “car” and “vehicle” will have similar embeddings because they are closely related in meaning.

Embeddings are widely used in real-world AI applications:

• Semantic search (finding relevant results based on meaning, not exact words)

• Recommendation systems

• Clustering and classification

• Retrieval-Augmented Generation (RAG) with vector databases

By converting text into embeddings, AI models can compare, search, and retrieve information more intelligently, making applications more accurate and context-aware.

A context window is the maximum amount of text an AI model can process at one time, measured in tokens.
It includes everything the model “sees” during a conversation your input, system instructions, and previous messages.

Think of it like the model’s working memory. If the input exceeds this limit, older or extra content gets truncated, which can affect the quality and continuity of responses.

Prompt engineering is the skill of designing effective inputs to get accurate, relevant, and high-quality outputs from AI models.
It plays a critical role in how well generative AI systems understand tasks and produce meaningful results.

In this section, you’ll learn how to structure prompts using proven techniques and patterns that improve AI performance across different use cases.

You’ll explore:

• Zero-shot and few-shot prompting to guide model behavior

• System vs user prompts for better control

• Role-based prompting to shape responses

• Chain-of-thought prompting for complex reasoning tasks

• Prompt templates for reusable and scalable workflows

• Prompt safety basics, including injection risks and mitigation

You’ll also learn practical best practices to:

• Write clear and structured prompts

• Reduce ambiguity and hallucinations

• Improve consistency and reliability

• Optimize outputs for real-world applications

By the end of this section, you’ll be able to design prompts that make AI systems more accurate, controllable, and effective across a wide range of tasks.

Retrieval-Augmented Generation (RAG) is a technique that enhances AI models by combining them with external data sources.
Instead of relying only on what the model was trained on, RAG allows it to retrieve relevant information from documents, databases, or knowledge bases before generating a response.

Why RAG is Important

Traditional LLMs:

• Have limited knowledge (training cutoff)

• Can produce hallucinations (incorrect answers)

• Don’t have access to your private or real-time data

RAG solves these problems by bringing in fresh, accurate, and domain-specific information.

How RAG Works (Simple Flow)

1. User asks a question

2. The system searches relevant data using embeddings + vector database

3. Retrieves the most relevant information

4. Sends that context to the AI model

5. The model generates a grounded, accurate response

Real-World Use Cases

• Document Q&A systems

• Customer support chatbots

• Enterprise knowledge assistants

• Research and analysis tools

In Simple Terms

RAG = Search + AI Generation

It allows AI to move from guessing to answering based on real data, making applications more reliable and useful.

A vector database is a specialized database designed to store and search embeddings—numerical representations of data based on meaning.
Unlike traditional databases that rely on exact matches, vector databases enable semantic search, meaning they find results based on similarity and context.

Why Vector Databases Matter

In generative AI applications, we often need to:

• Search documents by meaning, not keywords

• Retrieve relevant information quickly

• Support systems like Retrieval-Augmented Generation (RAG)

Vector databases make this possible by comparing how close embeddings are in vector space.

How It Works (Simple Idea)

1. Text (documents, queries) is converted into embeddings

2. These embeddings are stored in a vector database

3. When a query is made, it is also converted into an embedding

4. The database finds the most similar vectors

5. Relevant results are returned to the AI model

Common Use Cases

• Semantic search engines

• Document retrieval systems

• Recommendation systems

• Chatbots with memory or knowledge base

In Simple Terms

A vector database helps AI find the most relevant information based on meaning, not just matching words, making applications smarter and more accurate.

Context engineering is the process of designing and structuring the input given to an AI model to produce more accurate, relevant, and reliable outputs.
It goes beyond simple prompting and focuses on what information the model receives and how it is organized.

Why Context Engineering Matters

LLMs don’t “know” everything in real time—they rely entirely on the context you provide.
Poor or incomplete context leads to:

• Incorrect or vague answers

• Hallucinations

• Inconsistent responses

Well-designed context improves:

• Accuracy

• Relevance

• Consistency

What Goes Into Context

Context can include:

• Clear instructions (system prompts)

• User input

• Examples (few-shot learning)

• Retrieved data (from RAG systems)

• Memory from previous interactions

Key Techniques

• Structuring prompts clearly

• Providing relevant background information

• Limiting unnecessary data

• Using retrieved knowledge effectively

• Managing context within token limits

Context grounding is the process of ensuring that an AI model’s response is based on specific, relevant, and trusted information rather than general knowledge or assumptions.
It helps the model stay accurate, consistent, and aligned with the given data.

Why Context Grounding Matters

Without grounding, AI models may:

• Generate hallucinated or incorrect answers

• Provide generic responses

• Ignore important domain-specific information

Grounding ensures responses are tied to real, provided context.

How Context Grounding Works

• Supplying relevant documents or data

• Using techniques like Retrieval-Augmented Generation (RAG)

• Including clear instructions and constraints

• Limiting responses to the provided information

Real-World Examples

• Answering questions based on company documents

• Customer support using internal knowledge bases

• Legal or medical information systems

• Research assistants using specific datasets

In Simple Terms

Context grounding means making AI answer based on facts you provide, not guesses—leading to more reliable and trustworthy results.

Generative AI (GenAI) is a subset of Artificial Intelligence (AI) that focuses on creating new content such as text, images, code, audio, and more.
To understand where it fits, it’s important to see the relationship between AI, Machine Learning (ML), and Generative AI.

The Hierarchy

• Artificial Intelligence (AI) is the broad field focused on building systems that can perform tasks requiring human intelligence.

• Machine Learning (ML) is a subset of AI where systems learn patterns from data instead of being explicitly programmed.

• Generative AI (GenAI) is a further subset of ML that specializes in generating new content based on learned patterns.

How It Works

Generative AI models, such as Large Language Models (LLMs), are trained using machine learning techniques on massive datasets.
They learn patterns, structure, and relationships in data, which allows them to generate meaningful and coherent outputs.

Examples of GenAI

• Text generation (chatbots, content creation)

• Image generation

• Code generation

• Audio and video creation

AI safety refers to the practices and principles used to ensure AI systems behave reliably, ethically, and as intended without causing harm.
It focuses on making AI outputs accurate, secure, and aligned with human values.

Why AI Safety Matters

Generative AI systems can sometimes:

• Produce incorrect or misleading information (hallucinations)

• Reflect biases present in training data

• Be vulnerable to prompt injection or misuse

AI safety helps reduce these risks and builds trust in AI applications.

AI agents and chatbots both interact with users, but they differ in capability and purpose.

Chatbots primarily respond to queries, while AI agents can plan, take actions, use tools, and handle complex, multi-step tasks autonomously.

Agentic AI refers to AI systems that can act autonomously to achieve goals by planning, making decisions, and using tools.
Unlike traditional AI that only responds to prompts, agentic AI can take actions, adapt to tasks, and operate with minimal human input.

Key Concepts

• Goal-oriented behavior - works toward a defined objective

• Planning & reasoning - breaks tasks into steps

• Tool usage - interacts with APIs, data, or external systems

• Memory - retains context for better decision-making

Why It Matters

Agentic AI enables:

• Automation of complex workflows

• Intelligent assistants beyond chatbots

• Scalable, real-world AI applications

In Simple Terms

Agentic AI is about building AI systems that don’t just respond—but act, decide, and complete tasks on their own.

An AI agent is a system that can perceive information, make decisions, and take actions to achieve a specific goal.
Unlike simple AI models that only respond to prompts, an AI agent can plan tasks, use tools, and operate autonomously.

Core Components of an AI Agent

• Input (Perception): Receives user queries or data

• Reasoning/Planning: Decides what steps to take

• Memory: Stores past interactions or knowledge

• Tools/Actions: Interacts with APIs, databases, or external systems

• Output: Delivers results or completes tasks

How It Works (Simple Flow)

1. Receives a goal or instruction

2. Breaks it into smaller steps

3. Uses tools or data to execute tasks

4. Stores relevant information (memory)

5. Produces the final result

In Simple Terms

An AI agent is like a smart assistant that can think, plan, and act to complete tasks—not just respond to questions.

In Generative AI, models can be either hosted (cloud-based) or open-source (self-hosted), each with different advantages and trade-offs.

Hosted models are provided via APIs by companies and run on the cloud. They are easy to use, scalable, and require no setup, but may involve costs and data privacy considerations.

Open-source models can be run locally or on your own infrastructure. They offer more control, customization, and better privacy, but require setup, hardware resources, and maintenance.

Key Differences

• Ease of use: Hosted models are simpler; open-source requires setup

• Cost: Hosted is pay-as-you-go; open-source may reduce long-term costs

• Privacy: Open-source offers more control over data

• Performance: Hosted models often provide higher performance out-of-the-box

In Simple Terms

Hosted models are convenient and ready to use, while open-source models give you control and flexibility. Choosing the right one depends on your needs, budget, and use case.

Choosing between hosted and open-source AI models often comes down to cost-but the answer depends on your usage and setup.

Hosted models use a pay-as-you-go pricing model based on tokens or API calls. They have low upfront cost and no infrastructure needs, making them ideal for small projects or startups.

Open-source models are free to use, but require hardware (CPU/GPU), setup, and maintenance, which can increase initial costs.

Cost Comparison

• Short-term / low usage: Hosted models are usually cheaper

• High usage / long-term: Open-source can be more cost-effective

• Infrastructure costs: Open-source requires investment in hardware or cloud setup

• Maintenance: Open-source may need technical effort and time

In Simple Terms

Hosted models are cheaper to start, while open-source models can be cheaper at scale-if you have the resources to run them.

Choosing between hosted and open-source AI models depends on your use case, budget, technical skills, and data requirements.
There is no one-size-fits-all solution the right choice depends on what you’re building.

Key Factors to Consider

• Ease of use: Hosted models are quick to start; open-source requires setup

• Cost: Hosted is pay-as-you-go; open-source may be cheaper at scale

• Privacy & security: Open-source offers more control over sensitive data

• Performance: Hosted models often provide better out-of-the-box results

• Customization: Open-source allows deeper control and fine-tuning

When to Choose What

• Choose hosted models for rapid development, prototypes, and scalability

• Choose open-source models for privacy, control, and long-term cost optimization

In Simple Terms

Pick hosted models for speed and simplicity, and open-source models for control and flexibility based on your project needs and resources.

AI agents can be categorized into different types based on how they perceive information, make decisions, and act to achieve goals.
Understanding these types helps you choose the right approach when designing intelligent systems.

Common Types of AI Agents

• Reactive agents: Respond directly to inputs without memory

• Goal-based agents: Act to achieve specific objectives

• Utility-based agents: Optimize decisions based on outcomes or value

• Learning agents: Improve performance over time using data

• Multi-agent systems: Multiple agents working together or collaborating

Why This Matters

Different types of agents are suited for different tasks from simple automation to complex decision-making systems.

In Simple Terms

AI agent types define how smart and capable a system is ranging from simple responders to intelligent systems that learn, plan, and collaborate.

Memory is a key component of AI agents that allows them to store, recall, and use past information to make better decisions.
Without memory, agents can only respond to the current input, limiting their ability to handle complex or multi-step tasks.

Types of Memory in AI Agents

• Short-term memory: Keeps track of recent interactions within a session

• Long-term memory: Stores information over time, such as user preferences or knowledge

• Working memory: Holds relevant data temporarily for reasoning and task execution

Why Memory Matters

• Enables context-aware conversations

• Improves decision-making and personalization

• Supports multi-step task execution

• Reduces repetitive inputs

In Simple Terms

Memory allows AI agents to remember and learn from past interactions, making them smarter, more useful, and capable of handling real-world tasks.

AI agent architecture defines how different components of an agent are organized and work together to perceive, decide, and act.
It provides the blueprint for building intelligent, goal-driven systems.

Key Components of Agent Architecture

• Input (Perception): Receives user queries or data

• Reasoning/Planning: Decides what actions to take

• Memory: Stores and retrieves relevant information

• Tools/Actions: Interacts with APIs, databases, or external systems

• Output: Delivers results or completes tasks

How It Works

An AI agent processes input, uses reasoning and memory to plan steps, interacts with tools if needed, and generates outputs to achieve a defined goal.

Why It Matters

A well-designed architecture enables:

• Better decision-making

• Scalability and flexibility

• More reliable and efficient AI systems

In Simple Terms

Agent architecture is the structure that allows an AI system to think, act, and solve problems effectively.

Agentic tools are frameworks and platforms that help you build, manage, and deploy AI agents efficiently.
They provide ready-to-use components for handling tasks like memory, tool integration, and workflow orchestration.

What You’ll Learn

• Overview of popular agentic tools and frameworks

• How these tools simplify building AI agents

• Integrating APIs, data sources, and external tools

• Managing workflows, memory, and reasoning

Why Agentic Tools Matter

Instead of building everything from scratch, these tools help you:

• Develop faster

• Scale applications easily

• Focus on solving real-world problems

In Simple Terms

Agentic tools are like building blocks that make it easier to create powerful AI agents without handling all the complexity yourself.

Microsoft Copilot is an AI-powered assistant designed to enhance productivity and automate everyday tasks across applications and workflows.
It integrates advanced language models with tools you already use, helping you work faster and smarter.

What You’ll Learn

• What Microsoft Copilot is and how it works

• Key features and capabilities

• How it integrates with tools and workflows

• Common use cases for productivity and automation

Why It Matters

Copilot enables you to:

• Automate repetitive tasks

• Generate content and insights quickly

• Improve efficiency in daily work

• Build smarter, AI-driven workflows

In Simple Terms

Microsoft Copilot is like a smart assistant that helps you write, analyze, automate, and get more done with less effort.

In this hands-on project, you’ll build a Sales and Revenue Analyzer agent using Microsoft Copilot to turn raw data into actionable business insights.
You’ll learn how to design an AI-driven workflow that can analyze sales data, identify trends, and generate meaningful reports automatically.

What You’ll Learn

• How to design an AI agent using Copilot

• Processing and analyzing sales and revenue data

• Generating summaries, insights, and reports

• Automating business analysis workflows

Key Outcomes

• Identify revenue trends and patterns

• Generate data-driven insights instantly

• Build a practical business-focused AI agent

• Improve decision-making with AI automation

In Simple Terms

You’ll create an AI agent that acts like a business analyst—analyzing sales data and delivering insights automatically using Copilot.

In this hands-on project, you’ll build a Product Analyzer using the ChatGPT UI to transform raw product data into meaningful insights.
You’ll learn how to structure inputs and prompts to evaluate products, extract key features, and generate summaries automatically.

What You’ll Learn

• Designing effective prompts for product analysis

• Processing product descriptions and user inputs

• Extracting key insights, pros/cons, and summaries

• Building a simple, interactive analysis workflow using ChatGPT UI

Key Outcomes

• Automate product evaluation

• Generate clear and structured insights

• Improve decision-making with AI assistance

• Create a reusable analysis workflow

In Simple Terms

You’ll build an AI-powered tool that analyzes products and turns raw information into easy-to-understand insights using ChatGPT.

Llama is a family of open-source large language models developed by Meta that enables developers to build powerful AI applications with more control and flexibility.
Unlike closed models, Llama allows you to run, customize, and deploy AI systems in your own environment.

What You’ll Learn

• What Llama is and why it matters in AI

• Key features of Meta’s open-source LLMs

• How Llama compares to proprietary models

• Why open-source LLMs are important for innovation

Why It Matters

Llama enables you to:

• Build private and secure AI applications

• Customize models for specific use cases

• Run AI locally or on your own infrastructure

• Reduce dependency on closed AI systems

In Simple Terms

Llama is Meta’s open-source AI model that gives developers more freedom to build, customize, and deploy intelligent applications.

In this lesson, you’ll explore how Llama models can be used to build real-world local AI applications and private assistants.
You’ll understand how open-source LLMs enable secure, offline, and customizable AI solutions without relying on external APIs.

What You’ll Learn

• Real-world use cases of Llama models

• Building local AI applications with open-source LLMs

• Creating private AI assistants for personal or business use

• Benefits of running AI models locally (privacy, control, cost)

Key Outcomes

• Design privacy-focused AI applications

• Build offline or local chatbot systems

• Understand practical deployment scenarios for Llama

• Reduce dependency on cloud-based AI services

In Simple Terms

You’ll learn how to use Llama to build your own private AI apps and assistants that run locally, giving you full control over your data and models.