From Java Dev to AI Engineer: Spring AI Fast Track

Explore message roles in large language models—user, system, assistant, and function—and how Spring AI leverages them to craft better prompts and responses for enterprise apps.

Explore how Spring AI represents and sends system, user, and other message roles to LLMs using the prompt API, chat client, and role-specific constructors.

Explore how defaults in spring ai preconfigure chat requests with defaultSystem and defaultUser messages. Learn to wire a chat client bean in a config class and override defaults for contexts.

Leverage prompt templates in Spring AI to construct complex, reusable prompts with placeholders stored in resources; load with @Value and customize via @Param, improving maintainability and separation of concerns.

Explore prompt stuffing to give LLMs the needed context for accurate answers in simple scenarios, and note in-context learning and RAG as scalable options for large data.

Understand why prompt stuffing can't feed large data into llms due to token limits and billing, and learn how context windows, tokenization, and tokens shape input size and pricing.

Explains how the Spring AI framework uses advisors as interceptors in the prompt flow to perform pre-processing, post-processing, logging, and auditing for cross-cutting concerns.

Configure default advisor method and custom advisors for spring ai by using chartClientBuilder, log requests and responses with SimpleLoggerAdvisor, and implement CallAdvisor and StreamAdvisor for normal and streaming llm interactions.

Build a custom token usage advisor in Spring AI to log llm token usage and pricing per request, wiring it into chart client config.

Explore chat options in Spring AI to customize language models for enterprise applications, including model selection, frequency and presence penalties, temperature, top P, top K, stop sequences, and max tokens.

Configure chat behavior in spring ai by using chatoptions and its openaichatoptions implementation, tuning model, temperature, and tokens to govern llm chat interactions.

Explore how to extract LLM responses with content(), inspect metadata via chatResponse and chatClientResponse, and shape JSON or Pojo outputs using entity, while embracing stream() for real-time results.

Implement streaming AI responses in real-time with a Spring stream() API backed by Flux, enabling non-blocking delivery of LLM outputs and live browser demonstrations.

Explore how Spring AI's structured output converter adds formatting instructions and converts LLM responses into structured data like JSON or Java objects, enabling easy integration with business logic.

Learn how to structure llm outputs into a pojo CountryCities using spring ai, build a structured output controller, and expose a rest api that returns a RFC 8259 JSON payload.

Demonstrates using bean, list, and map output converters in spring ai to shape llm responses via the entity() method, returning list or map structures for rest APIs.

Learn to map an LLM response to a list of POJO objects using Spring's ParameterizedTypeReference, overcoming generics erasure with anonymous subclassing for runtime type retention.

Trace the roots of generative AI from artificial intelligence to large language models, exploring embeddings, vector stores, vector databases, and the roles of NLP and data science.

Explore the ai family from artificial intelligence to machine learning and deep learning, then examine natural language processing and generative ai, including llms and image generation.

Explore the two broad categories of generative AI models—large language models for text and diffusion models for images, video, and audio—along with training basics and enterprise relevance.

Explore how large language models learn from billions of words through unsupervised training, becoming text experts capable of predicting next words with vast compute.

The core job of a large language model is to guess the next word, acting as a smart autocomplete, while ChatGPT wrappers provide a user friendly interface and end-of-prompt cues.

Learn how large language models convert input text into tokens that can be words, parts of words, or characters, and how token IDs and vectors drive model outputs.

Understand how large language models build a model vocabulary during training, mapping words to token IDs and vectors. See why tokenization uses subwords for unknown words and vocabulary size matters.

Explore how embeddings and vectors teach AI models the meaning of words through high-dimensional token representations and clustering of concepts like king, queen, and royalty.

Discover how words convert to embeddings in a high dimensional vector space with thousands of dimensions capturing gender and royalty, enabling arithmetic like king minus man plus woman equals queen.

learn how static embeddings use fixed vectors and dot-product similarity to gauge word relatedness, and how projection and attention layers enable contextual, dynamic representations for better meaning.

Explore how positional embeddings augment input embeddings in transformer-based LLMs, enabling parallel token processing to capture word order and resolve meaning through attention layers.

Discover how the attention layer in LLM transformers uses context to resolve word sense and ambiguity, guiding focus to key words like deposit and river.

Explore the stateless nature of LLMs and how they forget past chats. See how Spring AI enables memory across conversations with chat memory configurations and a practical demo.

Explore how Spring AI's chat memory preserves context by storing and appending past messages, using strategies like last N messages and time windows with in-memory or jdbc repositories.

Configure chat memory strategies using the chat memory interface to store and retrieve messages. Explore memory advisors: MessageChatMemoryAdvisor, PromptChatMemoryAdvisor, and VectorStoreChatMemoryAdvisor, to inject context into prompts and support long-term conversations.

Configure chat memory for a rest api with spring ai by wiring chat memory and advisors into the chat client. It highlights a default conversation id and its drawbacks.

Implement per-user memory for language models by using a unique conversation id from a username header to keep each chat separate.

Customize Spring AI by configuring MessageWindowChatMemory with maxMessages (for example 10) and injecting a JdbcChatMemoryRepository to override default chat memory behavior.

Balance max messages with the model's context window in Spring AI apps, choosing suitable models, monitoring token usage, and using vector store memory to keep relevant history while managing costs.

Explore how the Rag framework enhances llm responses by feeding relevant context from a vector database–backed knowledge base, addressing private data limitations and enabling personalized, accurate results.

Master the rag flow—retrieval, augmentation, and generation—to load organization documents into a vector store, retrieve relevant document chunks, augment prompts, and generate accurate responses with Spring AI.

Discover how vector stores convert unstructured data into vectors and embeddings for semantic search, enabling retrieval augmented generation with LLMs.

Demonstrates RAG in a Spring Boot AI app by loading random sentences into a Qdrant vector store, converting them to documents, and exposing a RAG API with a system prompt.

Learn to perform vector store searches with a search request, tune topk and similarityThreshold, and generate prompts from similar documents to drive a chat-backed RAG flow.

Demonstrates rag using a random data loader and vector store to test rest api prompts, showing how similar context under the system prompt informs responses like 'i don't know'.

See how to use RAG to query an EasyBytes HR policy document, chunk content for a vector store, and tailor prompts to reveal hours, leaves, and other policy details.

Split large documents into 100-token chunks with TextSplitter and Apache Tika, then index them in a vector store for targeted retrieval while tuning top k to reduce prompt tokens.

Explore retrieval augmented generation with the RetrievalAugmentationAdvisor to query a vector store, tune top k and similarity thresholds, and automatically populate the system prompt in REST APIs.

Define a RAG flow with web search by implementing a DocumentRetriever interface and using a vector store option, then fetch latest information through Tavily search APIs.

Build a web search rag flow by implementing a web search document retriever that calls Tavily, processes top results, and feeds context to the llm.

Implement advanced rag flows by applying pre-retrieval and post-retrieval steps to transform, filter, rewrite, translate, and summarize queries for concise results in a vector store and LLM pipeline.

Explore advanced RAG with post-retrieval processing to mask emails and other sensitive information and refine retrieved documents before feeding them to the LLM.

The Spring AI framework computes document embeddings via the OpenAI embedding model and stores them in a vector store like Qdrant, enabling rag flow with batching to handle many documents.

Discover how semantic caching reduces LLM costs and latency by using embeddings and vector stores to match meaning, not exact text, with sensible similarity thresholds and TTL.

Explore semantic caching with Redis and Qdrant in a Spring Boot setup. Learn to configure Redis client, embedding model, and similarity thresholds to reduce LLM calls via semantic cache advisor.

Learn to implement semantic caching with a Qdrant vector store for RAG workflows, comparing it to Redis and tuning similarity thresholds for efficient cache hits.

Enable tool calling in Spring AI apps to fetch real-time data and perform actions via Java methods, empowering LLMs to read live data and book tickets or send emails.

Explore why LLMs cannot answer real-time questions and learn to implement tools calling in a Spring AI app by building a time REST API with a TimeController and TimeChatClientConfig.

Build and expose time tools in Spring AI using @Tool and @ToolParam. Configure and test these tools to return the current local time or time in a specified time zone.

Examine how Spring AI calls tools internally: user prompts trigger LLM tool calls guided by tools details, executed by DefaultToolCallingManager, with finish reasons tool calls or stop.

Build tools that let LLMs fetch real-time data and perform actions, such as creating and querying help desk tickets in a database, using Spring Data JPA and a service layer.

This lecture demonstrates configuring a dedicated helpdesk chat client with a custom system prompt, building tools for database interactions, and exposing a tool-calling API using tool context for user data.

Watch llms talk to databases through a tool calling demo, creating and querying HELPDESK_TICKETS in an h2 database, with rest api testing via postman.

Enable the returnDirect flag on the @Tools annotation to bypass the LLM and send tool outputs directly to the client, reducing latency for the end user.

Explains how runtime tool errors become ToolExecutionException in Spring AI, and how enabling a true boolean processor sends details to the client, demonstrated with a ticket status scenario.

Trace the evolution of generative ai from llm-based knowledge to rag and tool calling. Explore how Spring AI framework enables building intelligent applications and ai agents that act autonomously.

Understand the model context protocol (MCP) introduced by anthropic and how a dedicated MCP server exposes tools to LLMs, enabling external access and modular gen AI workflows.

Explore the MCP protocol that standardizes how apps provide context to LLMs by centralizing tools, prompts, and resources on a secure MCP server and coordinating host, client, and server roles.

Build a Spring AI app as an MCP host and client, connect to an MCP server using stdio transport, and wire tool callbacks provided by ToolCallbackProvider for the llm-driven chat.

Explore the MCP inspector to quickly understand the tools exposed by an MCP server and test them, using the npx @modelcontextprotocol/inspector workflow after installing node.

Demonstrates integrating a spring ai application with the GitHub MCP server to manage repositories via Docker, personal access tokens, and natural language prompts.

Build and test an MCP server using stdio, generate Java jar with Maven, and integrate with an MCP client to validate tool invocation via a REST API and username header.

Master dynamic, request-level filtering of MCP tools in Spring AI to expose only needed tools per scenario, reducing token consumption and improving LLM performance.

Learn how MCP logging streams server events to the client, making a black-box server observable. Implement a client log bridge using McpLogging to capture info, debug, and error messages end-to-end.

Explore how MCP sampling lets an MCP server offload intelligent tasks to client applications with an LLM, enabling a callback-driven tool execution without server-side models.

Showcase MCP sampling by implementing HelpdeskSamplingProvider with @McpSampling, using OpenAiChatModel to forward prompts to an LLM, and exposing summarize-tickets to return tool-generated summaries.

Explore the MCP annotation family in Spring AI, enabling declarative server and client operation handlers with @McpTool, @McpResource, @McpPrompt, and @McpComplete, while reducing boilerplate via the MCP Java SDK.

Learn how to unit test Spring AI applications by using evaluators to assess non-deterministic LLM responses, and validate prompts and answers through the ChatClient workflow.

Build trustworthy ai in a spring ai project by using relevancy and fact-checking evaluators to assess LLM responses in chat and rag contexts, including prompt-stuffing from HRPolicy.st, with unit testing.

Demonstrates using the relevancy evaluator to test the chat controller's responses and keep LLMs on topic, validating non-blank results and a true isPass with a minimum score.

Integrate the Spring AI fact-checking evaluator into production chat handling to validate LLM outputs at runtime, throwing InvalidAnswerException when responses fail the check.

Enable runtime validation by retrying invalid LLM responses with spring retry and @Retryable, plus a recover method. Favor fact checking evaluators like Bespoke Minicheck to reduce hallucinations.

Enable observability in a Spring AI application by exposing metrics and tracing via the actuator, and monitor AI operations across logs, metrics, and tracing.

Expose spring ai metrics to Prometheus and explore its ui at localhost:9090, then monitor token usage with scrapes and dashboards, and plan Grafana integration for rich visuals.

Set up Grafana with docker, connect to Prometheus in the spring-ai network, and persist dashboards with grafana-storage while building token usage and tool metrics dashboards on port 3000.

Expose tracing information from your Spring AI application using OpenTelemetry and Jaeger; visualize traces in Jaeger UI and Grafana to locate bottlenecks across llm models and layers.

Explore how Spring AI transcribes audio into text using a transcription model for chat and case workers. Also learn to generate audio from text and create vtt subtitles with options.

Explore text-to-speech workflows in Spring AI: inject a speech model, build a speech prompt with options like voice and speed, and generate and save mp3 audio from an LLM response.