AI ENGINEERING & LLM MASTERCLASS

Master prompt engineering by structuring prompts with clear system, user, and assistant roles, using zero-shot, one-shot, and few-shot approaches, and adopting XML-tagged sections for consistent, JSON-ready outputs.

Learn advanced prompting techniques to improve model reasoning, including chain of thought, self-consistency, tree of thought, and least-to-most, with guidance on use cases and cost considerations.

Learn to produce reliable structured output with a three-level approach: simple prompts, guaranteed JSON syntax, and level 3 validation using Instructor and Pydantic for production.

Automate evaluation to detect prompt regressions with eval suite of test cases, using a GPT-4.0 judge to score accuracy, completeness, and format, run via GitHub Actions on each prompt change.

Master prompt versioning and management with Jinja2 templating and a prompt library, preventing audit, concurrency, and drift issues through versioned prompts and traceability.

Explore the OpenAI API deep dive, mastering streaming, function calling, and batch processing, plus a robust prompt library with save, load, promote, and run for scalable AI apps.

Master four patterns in one file with a complete tool calling loop in the Anthropic Claude API, including explicit parameters, streaming, and function tools.

Build a thin gateway to orchestrate multiple providers, enabling failover, routing, cost tracking, and seamless model switching via LightLM. Route tasks to cheap models to cut costs by 55%.

Learn to accelerate batch API workflows by using async I/O and semaphores to run concurrent requests, apply exponential backoff for rate limits, and achieve 8–15x speedups.

Apply caching strategies to cut API calls and costs by using exact and semantic caching. Leverage embeddings and cosine similarity with a 0.95 threshold to reuse answers, especially for GPT-4.0.

Implement observability and debugging for AI systems by instrumenting OpenAI calls, emitting json lines, and tracking latency, model, prompts, and token usage to diagnose issues quickly.

Explore the infrastructure behind RAG, including vector databases, embeddings, and semantic search, and learn how embedding-based search overcomes keyword limitations in production AI.

Explore similarity search for embeddings: use cosine similarity for text, note dot product for normalized vectors, avoid Euclidean distance, and apply exact vs approximate search with HNSW in vector databases.

Identify a practical framework to pick vector databases: development with ChromaDB, production with Pinecone or PgVector. Learn how metadata filtering enables semantic search within specific data subsets.

Build a production-ready ChromaDB document store with metadata, embeddings, and unique IDs, using upsert ingestion and a persistent SQLite backend, and explore collection structure and query operators.

Learn four chunking strategies to improve retrieval in RAG systems: fixed-size, recursive character, semantic, and document-specific chunking; use paragraph boundaries, overlaps, metadata, and evaluation to ensure complete context.

Learn how retrieval augmented generation uses document retrieval with embeddings and a vector database to ground answers, reduce hallucinations, and integrate with prompts, APIs, and data documents.

master naive rag by building a six-step pipeline—load, chunk, embed and store, embed query, retrieve top chunks, build prompt with context, and generate answer—and study its common failure modes.

Explore advanced retrieval techniques to reduce retrieval misses and boost recall, including multi-query retrieval, HIDE, parent-child retrieval, MMR, and reranking with cross-encoder models.

Assess and optimize a RAG system using the RAGAS framework with four lenses—faithfulness, answer relevancy, context precision, and context recall—turning improvement from guesswork into engineering.

Hybrid search merges semantic search and BM25 to handle exact string queries like SKU4421 reorder policy, surface the right document first using reciprocal rank fusion.

Implement corrective RAG by adding a retrieval quality check between retrieve and generate, using a quality evaluator and a three-way branch to handle good, ambiguous, or failed retrievals.

learn how to deploy production-ready rag architectures that run reliably and efficiently, handle failures gracefully, auto-update with document changes, and keep a sensible cost profile in production.

Explore LCEL chains and runnable components connected by pipe, including prompts, models, parsers, and retrievers. Learn five patterns: sequential, parallel, RAG chain, routing with branch, and fallbacks.

Explore memory systems for llm prompts by applying buffer memory, summary memory, summary buffer, and token-ware approaches to manage conversation history, token costs, and context window efficiency for production readiness.

Explore document loaders and text splitters, including PyMuPDF for PDFs, unstructured for complex layouts, and web loaders, with the Inspect Docs tool, smart splitting, and metadata propagation for reliable RAG.

Learn how Lang Chain retrievers plug into RAG chains as composable runnables, starting with the vector store retriever and evolving with multi-query, contextual compression, and self-query options.

Compare Langchain and instructor output parsers for structured data, use string, JSON, and Pydantic parsers, and apply retry-with-error within LCEL chains to produce validated, easily usable results.

Design safe, well-described tools for AI agents to query stock, lead times, and warehouse management system data, and enforce a three-tier safety framework with structured errors and parallel tool calls.

Master LangChain agents by using tool decorators, docstrings, and type hints to automate boilerplate and generate JSON schemas; enable OpenAI function calling and verbose debugging.

Master the model context protocol (MCP), an industry standard exposing tools once as an MCP server for universal access; Think HTTP for AI tools with tools, resources, and prompts.

Decide when fine-tuning is appropriate, prioritize prompt engineering and RAG, and use LoRa/QLoRa with a practical decision framework to shape domain tone.

Leverage HuggingFace PEFT with LoRa and QLoRa to fine-tune large Llama models on affordable hardware, using 4-bit NF4 quantization, targeted adapters, and scalable cloud resources.

Navigate the open-source models landscape, comparing LLAMA 3.x variants, Mistral and Mixtral, and Gemma 2, and weigh privacy, cost, and engineering needs for self-hosted deployments.

Master local LLM deployment with Ollama, a one-install, one-command tool that provides an OpenAI-compatible REST API, offline RAG, and zero API calls.

Master production serving with vLLM, the Berkeley open-source standard that achieves 10–20x throughput and 70–90% GPU utilization via continuous batching and efficient KV cache management, with an OpenAI compatible API.

Explore cost optimization for AI projects by implementing semantic caching, model routing, prompt compression, and batch API strategies, plus practical tests and cache hit rate insights.

Implement layered guardrails for production AI, with fast input checks for prompt injections and PII, and slow output checks for schema, length, faithfulness, and context overlap.

Explore deployment strategies for LLM systems, using shadow mode to test with real production traffic and compare outputs, and canary rollouts to monitor latency, cost, and errors while automating rollback.

Test llm systems with deterministic unit tests and a golden dataset of expert-verified input-output pairs; run prompt changes against it, and use field-level evalmodel scores to gate deployment in ci-cd.

Learn practical llm ops tooling to keep ai systems healthy and cost-controlled in production. Use LangSmith with LangChain for tracing prompts, responses, and latency; consider LangFuse for data residency.