Complete GenAI with Java & Spring AI: LLMs, RAG, AI Agents

Explore Spring AI's framework for integrating AI-powered services into Spring Boot, with auto-configuration, multi-provider support (OpenAI, HuggingFace, DeepSeek, Ollama), prompt templates, and RAG for AI in microservices.

Learn to obtain and configure your OpenAI API key for Spring AI projects, including generating, saving, and exporting the key to your environment.

Boot a Spring AI app with OpenAI, configure a chat endpoint and a chat client, apply one-shot prompts and system messages, and create reusable OpenAI chat beans for multiple models.

Develop a summarize endpoint at api/openai/chat/summarize in a Spring Boot app, injecting the OpenAI chat client and using a fluent prompt to summarize content. Apply a system prompt guardrail.

Switch from content to chatClientResponse to access the full response body, including tokens, model, and metadata. See how Spring AI builds an advisor chain and standardizes responses across LLM providers.

Leverage an in-place prompt template for secure and simplified summarization of meeting notes via a dedicated endpoint, passing meetingNotes as a parameter and using a template-driven user message.

Learn how to call the OpenAI API directly from Java using an HTTP client to post to v1/chat/completions, with system and user messages, and compare to Spring AI.

Learn to use multiple AI models in Spring AI by wiring distinct chat model implementations via qualifiers or dynamic configuration, enabling seamless switching between OpenAI and Vertex AI.

Implement a java doc comment generator for java code using google gemini via vertex ai. Switch between openai and vertex ai with a configuration variable to generate javadocs.

Learn how HuggingFace hosts open-source and community models, use them via the shared inference API or hub, and weigh latency, cost, privacy, and model loading considerations.

Explore building a programming helper with HuggingFace in Spring AI, including API key setup, shared versus dedicated endpoints, and configuring Qwen Coder 7b instruct for code generation.

Configure HuggingFace shared and dedicated inference endpoints in Spring AI via an OpenAI-compatible API using qwen 2.5 coder 7b instruct, then explore TGI and vLLM engines.

Learn about open source large language models and how to run them locally, covering model weights, open source licenses, source code, training data, and local tooling like Ollama or Docker.

Explore how Ollama runs local LLMs like a docker for your machine, with an OpenAI API layer, model filtering, and a CLI workflow (pull, list, run).

Demonstrates building an email drafting assistant with Spring AI and Ollama using the Mistral 7b model, including local chat setup, OpenAI-compatible API, and a custom Mistral model file.

Learn to run local LLMs with docker-module-runner in docker desktop, expose an OpenAI compatible API on port 12434, and build a LinkedIn post generator using Gemma3 with Spring AI.

Use OpenWebUI to chat with local LLMs via Ollama and Docker model runner. Configure docker commands, access the admin panel, and test models like ai/gemma3 and mistral.

Explore the spring ai chat client and chat model integration, including the advisor chain, deque, and fluent api, illustrating a modular OpenAI backend with stateless requests.

Explore Spring AI's advisor architecture, where a callAdvisor chain ends in the real AI model call via a deque order. Know the four message types: user, system, assistant, tool.

Explore streaming in Spring AI with text event stream, server-sent events, and flux, showing how to implement summarizeWithStreaming and stream responses from the OpenAI summarization endpoint.

Explore spring ai response types, including content and summarize methods, chat and chat client responses, and how context supports tool call results and structured output.

master structured output in Spring AI by returning Java objects, lists, or maps through entity methods and structured output converters; create summarization responses with action items, decisions, and error messaging.

Learn how to configure chat options in Spring AI, using imperative and declarative approaches, override model name and max tokens per request, and understand token and context window implications.

Shaping temperature influences LLM output by reshaping logits before softmax, impacting determinism and creativity. Use lower temperatures for deterministic results; higher temperatures add randomness and varied responses with practical guidance.

Explore top-p (nucleus sampling) and top-k for controlled randomness, compare with temperature, and apply stop sequences along with frequency and presence penalties to shape OpenAI style outputs.

Explore the advisor architecture in Spring AI, using an around chain of pre-request and post-response advisors, including SimpleLoggerAdvisor logging requests and responses.

Add Safeguard Advisor to the Summarizer flow, configure with sensitive words, enforce pre-model rejection, explore advisor order and logging, and prepare for custom error handling with ErrorWrappingAdvisor.

Add a custom error-wrapping advisor to the summarizer use case, placing it as the first, highest-precedence advisor in the chain to wrap downstream errors and return a JSON summarization.

Learn to add system prompt and validation advisors to guide a summarizer, update system messages, and enforce a guardrail within an advisor chain for strict summarization.

Add a custom validation advisor to the summarizer use case, implement input and output validation, enforce a 5000-character limit, and centralize error handling for graceful responses.

Master prompt engineering for large language models with zero-shot, one-shot, and few-shot prompting, chain-of-thought reasoning, background context, boundaries, examples, and iterative refinement.

Master prompt engineering basics: set system messages to define llm roles and guardrails for summarization tasks, generate multiple results, and control tokens and advisor usage across OpenAI and Spring AI.

Revisit prompt engineering best practices, showing how temperature, top-p, and top-k shape creativity in LLM responses. Learn to use prompt templates for secure, maintainable, parameterized prompts.

Explore prompt injection techniques and mitigations, including direct and indirect injections in AI agents, with hands-on demonstrations of jailbreaking and prompt leaking.

Explore jailbreaking as a prompt hacking technique in a Summarizer use case, bypassing safeguards via tool calling, and learn mitigations like validation and authentication to prevent prompt injection.

Explore prompt leaking as a prompt hacking technique where prompt injection reveals system or developer instructions, demonstrated via the summarizer endpoint, and learn guardrails to prevent it.

Explore advanced prompt hacking techniques, including context switching and dictionary attacks, with demonstrations of prompt injection and how sandwich defense and mitigations improve secure AI applications.

Learn prevention and mitigation of prompt hacking attacks by implementing input validation, sanitization, safeguard and validation advisors, output filtering, and system prompt guardrails.

Explore the fundamentals of generative AI and large language models, covering machine learning, deep learning, neural networks, learning types (supervised, unsupervised, reinforcement), and RLHF, with practical use cases.

Explore the fundamentals of generative AI and large language models, including NLP tasks like tokenization, sentiment analysis, translation, and question answering, powered by transformers and attention for scalable language understanding.

This lecture explains how tokenizers convert text to tokens and IDs, forming the vocabulary that powers LLMs and their embeddings as numeric vectors.

explain positional encoding in transformers, including absolute vs relative schemes, and how token embeddings plus position vectors create enriched inputs. outline contextual embeddings through self-attention, multihead attention, and feed-forward layers.

Explore how large language models predict the next token using logits and softmax in the next token generation loop, converting scores to probabilities, and how temperature and sampling shape outputs.

The lecture examines whether LLMs think like humans, showing they can simulate step-by-step reasoning while actually predicting tokens; they don’t understand or introspect their own reasoning.

Compare costs, benchmarks, and context windows to choose an LLM for your use case. Evaluate latency, privacy, and tool support across providers, including self-hosted and Spring AI options.

Explore how chat memory works for large language models using Spring AI. Compare in-memory and persistent memory options and implement memory with message, prompt, and vector store advisors.

Explore the OrderStatusHelper use case with in-memory chat memory in Spring AI. Implement a memory-backed order status tool, DTOs, and a concurrent map to track and update status.

Demonstrate an in memory chat memory implementation for an order status helper, showing per user memory, tool calls, and request headers that govern access and responses.

Demonstrates persisting chat memory in Postgres via JDBC in Spring AI, switching from in-memory to JDBC repository. Configures Docker Postgres and maintains last five messages for an order status helper.

Set up a pgVector-backed vector store in PostgreSQL to enable long-term memory for chat conversations, using similarity search with embeddings for contextual recall.

Demonstrate vector store memory in a Spring AI app, using long-term memory with pgvector, custom system prompts, and a hybrid recall approach to reliably retrieve order status.

Explore how PromptChatMemoryAdvisor compares with MessageChatMemoryAdvisor and VectorStoreChatMemoryAdvisor, showing how memory is injected into the system prompt as plain text for portable, cost-efficient chat memory management.

Compare vector store chat memory across models and providers, highlighting prompt chat memory advisor for consistent memory behavior, handling dimension changes and system prompts with multiple memory adapters.

Explore multimodality in Spring AI, enabling text-to-image, image-to-text, text-to-speech, and speech-to-text processing. The lecture introduces how Spring AI orchestrates multimodal inputs and outputs using the multimodality API.

Implement a visual compliance checker that converts product images to text, extracts metadata, and detects mismatches between image and description using Spring AI and OpenAI GPT-4o.

Create a marketing asset generator that converts text to image using spring ai and OpenAI dali 3, configuring image options and system prompts to return a url or base64 image.

Build a smart meeting assistant for a speech-to-text workflow that transcribes audio with whisper-1 and summarizes the transcription with a GPT-4o chat client to extract action items.

Create a text-to-speech ticket status notifier using Spring AI and OpenAI, turning ticket status (open, waiting on customer, processing, result) into mp3 audio with TTS-1 and GPT-4o.

Identify key LLM limitations that affect correctness, performance, and security, such as context limits and hallucinations. Preview mitigations and present retrieval augmented generation as the first advanced solution.

Explore engineering strategies to mitigate LLM limitations, including structured output, validation, prompt guarding, tool calling, versioning, and retrieval augmented generation.

Map LLM limitations to mitigations using memory and rack strategies. Implement grounding, tool calls, and citations to reduce hallucinations and stale knowledge.

Explore retrieval augmented generation with rack for large dynamic context, and imprompt context or prompt stuffing for static it support prompts; learn to manage noise, token use, and potential hallucinations.

Introduce RAC, retrieval augmented generation, and its use cases with embeddings, vector stores, and similarity search to supply relevant context for LLMs, reducing cost and hallucinations.

Implement the ingestion pipeline for the rack use case with a simple vector store and a question answer advisor, including creating five employee handbook markdown files and configuring rack properties.

Continue the employee handbook use case by building or loading a vector store with embeddings. Add metadata and chunks, leveraging the OpenAI embedding model and Spring AI for semantic retrieval.

Build the employee handbook retrieval pipeline by configuring an OpenAI rack chat client with a simple vector store and a question-answer advisor using markdown resources for context.

Explore how the question answer advisor retrieves documents via cosine similarity over embeddings in a vector store, augments prompts with context, and tunes top k and similarity threshold.

Explains byte-pair encoding and CL100K tokens, and how the token-text splitter chunks text for retrieval augmented generation. Aligns chunk size with model limits using 800 tokens as a practical standard.

Explore how chunks are embedded via the embedding model and saved in a simple vector store, while chunk size, top-k, similarity thresholds, and overlapping influence recall and retrieval.

Explore embeddings that convert text to fixed-length vectors, tokenize into subwords, apply self-attention to create sentence representations, and use cosine similarity in a vector store for semantic search.

Understand vector databases storing embeddings with text and metadata for fast approximate nearest neighbor search using HNSW and IVF, with pgVector and metadata filtering for scalable semantic search.

Learn semantic search with embeddings and vector stores, using cosine, euclidean, or dot product metrics to retrieve similar chunks. Apply quality boosters, pre-filtering, and hybrid BM25 approaches to improve relevance.

Implement the ingestion phase of an advanced RAG flow for internal QA, ingesting pdfs into pgvector and enabling a retrieval pipeline with advanced tools.

Configure two vector store beans for chat memory and rack with table names and config data; build a retrieval augmentation advisor and migrate token splitter to AI Promoter for rack.

Implement the rack ingestion service to power the ingestion pipeline that reads PDFs from a configured path and stores document chunks in the rack vector store.

Advance the rack ingestion pipeline by implementing get documents parts and get chunks with pdf readers, enriching metadata, and indexing embeddings into a pg vector store for advanced rag flow.

Build the internal qa rac retrieval pipeline with a vector store using top k and similarity threshold, plus a retrieval augmentation advisor and runtime metadata filtering.

Add a domain synonym transformer as a preprocessor in the retrieval pipeline, using configured synonyms to transform queries and trim polite prefixes and question marks.

Adapt the post processor to fetch neighbor chunks via chunk index and a radius, preserving context with overlap and deduplicating by source and index.

Implement neighbor stitching in the retrieval pipeline by building a filtered vector store search, merging and deduplicating target and neighbor documents to enrich context before the LLM prompts.

Add a citation header post-processor to the retrieval pipeline, extracting source and page numbers from metadata and enforcing header citations in LLM prompts.

In the retrieval augmentation advisor, implement a query expander that generates four paraphrase variants of the user query to boost recall via vector search, balancing latency and token usage.

Add a re-ranker as a post-retrieval step to the vector search pipeline, using a cross-encoder model via Cohere's re-rank API to rescore candidates and boost precision.

Learn to implement a Cohere reranker in the retrieval pipeline for an internal q/a use case. Build getScores, handle incomplete results, and surface top end documents via a post processor.

Enhance the ingestion pipeline with update and delete methods, enabling runtime vector store updates via a folder watcher and checksum-based deduplication.

Learn to implement a pdf watcher service as a Spring component with smart lifecycle, watching a directory and debouncing events to update the rack vector store at runtime using RAG.