AI Engineering Fundamentals

Explain the q-k-v mechanism of attention, showing how queries, keys, and values compute similarity, weight values into a weighted sum, and produce a soft lookup in self- and cross-attention.

Learn how attention masking controls what a transformer can attend to, using causal masks for generation and padding masks for batched sequences, applied before softmax.

Enforce left-to-right autoregressive generation with causal masking, preventing future-token access by masking the upper triangle of the attention matrix while allowing the current token on the diagonal.

Explore multi-head attention, where multiple attention mechanisms run in parallel, each learning distinct relationships like syntax, coreference, and formatting patterns, to enrich transformer representations for n-l-p tasks.

Learn how positional encoding injects sequence information into transformers, using sine-cosine or learned embeddings, and when to rely on simple encodings for text and sequential data.

Explore how feed-forward networks complement attention in transformers by applying a two-layer mlp to each token, expanding representations and enabling non-linear feature extraction.

Understand residual connections in transformers, where x plus F(x) creates a skip path around attention and feed-forward sublayers, preserving information and enabling gradient flow for deep stacking.

Explore encoder-decoder models, a classic sequence-to-sequence architecture that encodes input bidirectionally and decodes with cross-attention for translation, summarization, and rewriting.

Explore llm inference as runtime that turns prompts into outputs by tokenizing input, performing a forward pass to obtain next-token probabilities, and applying decoding strategies for latency, safety, and creativity.

Understand parameter scaling and its trade-offs, including increased parameters, layers, attention heads, and MLP size, with scaling laws guiding when to scale, optimize compute, and use R-A-G.

This module unpacks transformer machinery: attention, QKV projections, and positional encoding for GPT-style decoding. See how multi-head attention, residual connections, and layer normalization enable debugging and scalable LLMs.

Master tokenization, token IDs, and context windows; and experiment with temperature, top-k, top-p, stop sequences, and streaming in an interactive LLM playground to balance cost and quality.

Explore Byte-Pair Encoding (BPE) as a frequency-based tokenization method used by GPT, merging frequent character pairs to build a compact, efficient vocabulary.

Explore tokenizer vocabularies, the model’s dictionary of token pieces, including token IDs and subwords, and learn how fixed vocabularies shape cost, context window, and you can't add words at runtime.

Master the context window by budgeting tokens for instructions, retrieved context, and the answer, while balancing input size, history, and tool outputs to ensure reliable LLM performance.

Beam search maintains B candidate sequences, expands them at each step by cumulative log probability, and keeps the top beams to maximize likelihood and avoid premature greedy choices.

Compare multiple models with identical prompts to evaluate quality, speed, and cost, then apply a repeatable framework for model selection using OpenAI, Anthropic, and Ollama.

Explore tokenization, temperature, top-p and top-k controls in a hands-on llm playground, streaming outputs, and cross-model comparisons to optimize prompts, costs, and production decisions.

Design prompts as an interface contract with LLMs, detailing goals, context, constraints, inputs, and output formats like JSON or tables to ensure reliable results.

Learn how system prompts shape model behavior in prompt engineering. Define identity, behavior rules, constraints, and output format, and explore role prompting for targeted perspectives.

Explore context injection in ai apps, how untrusted retrieved content can influence models, and strategies to label untrusted data, delimit content, and mitigate risks in rag and tool-augmented agents.

Explore zero-shot prompting, giving clear instructions without examples to power fast baselines in classification, summarization, rewriting, with JSON output schemas for precise results.

Explore chain-of-thought prompting and self-consistency to reveal verifiable reasoning in multi-step problems, compare direct prompting with chain-of-thought prompting, and apply majority voting for higher accuracy in ai engineering fundamentals.

Learn self-consistency, a prompting technique that runs the same prompt multiple times with randomness to derive a consensus answer for reasoning-heavy tasks.

Unlock the ReAct pattern—reason, act, and observe—building a multi-step agent that uses tools like a calculator and lookup. Learn prompt engineering, tool orchestration, and adaptive reasoning for complex questions.

Leverage prompt chaining to break complex tasks into sequential prompts, where each step handles a piece of the task and passes results forward to improve outputs.

Explore prompt chaining: decompose tasks into a three-step pipeline—extract key points, classify sentiment with a score, and deliver a summary and recommendations.

Iteratively optimize prompts with explicit constraints and testing to produce consistent, structured outputs, minimize drift, and apply in customer support and legal ops.

Apply version control to prompts, treating them as production code, tracking changes, versions, and the full bundle to reproduce outputs and roll back safely.

This document data extractor lab builds a robust pipeline that outputs validated JSON from messy unstructured text, across invoices, using a defined schema, JSON mode, and Pydantic validation.

Apply reliable prompt engineering with roles, system prompts, templates, and schema-first outputs to ensure secure, production-ready prompts. Leverage embeddings, document processing, and guarded retrieval to manage untrusted input and injection.

Master the RAG pipeline from embeddings and document preparation to chunking and indexing with HNSW, enabling fast retrieval via cosine similarity in a vector database.

Explore how vector databases store embeddings and enable fast nearest-neighbor search for semantic search and rag pipelines, complementing relational databases and guiding chunking quality for reliable retrieval.

Explore hnsw, a hierarchical navigable small-world graph for fast, approximate nearest neighbor search in embeddings. Tune ef search, M, and ef construction to balance speed and recall in vector databases.

Learn how chunk overlap preserves boundary context in RAG, improving retrieval for contracts, manuals, and policies by repeating surrounding text across chunks.

Explore how chunk size and overlap impact retrieval quality in rag pipelines, using a hands-on mini-lab to compare character-based chunking with overlapping chunks and observe effects on embeddings.

Extract structured metadata from messy documents to create clean json records with fields like date, author, and topics before embedding and indexing, enabling filtering, routing, and secure retrieval alongside embeddings.

Attach structured metadata to each chunk and use metadata filtering to sharpen semantic search results. Compare filtered and unfiltered queries to improve retrieval relevance and recall in production rag systems.

Turn your built document pipeline into a hire-ready portfolio with four connected demos: module 5 semantic search, module 6 RAG system, module 7 tool-using agent, module 10 production API.

Learn how context retrieval powers RAG systems by selecting relevant private data to ground LLM answers. Explore hybrid retrieval with vector search and BM25 to avoid noise and missed matches.

Learn how context injection shapes AI outputs in RAG, and how to label untrusted content, delimit it, and apply mitigations to prevent misuse.

Explore how source citation grounds rag answers by attaching evidence from documents, urls, page numbers, and text snippets to verify and audit every response.

Build an end-to-end RAG pipeline that retrieves relevant chunks, embeds and chunks documents, stores them in Chroma DB, injects context, and generates grounded answers with citations.

Build an end-to-end rag pipeline with retrieval, context formatting, and citation-based generation. Emphasize grounded results, proper source citations, and the agent pattern for multi-step tasks.

Learn function calling to bridge language models and real actions by defining tools with JSON schema and executing Python functions in a round trip, while the LLM never executes code.

Tool execution lets an LLM act by running external actions like API calls or queries, while the application validates arguments with schemas and requires human confirmation for destructive actions.

Build a framework-free python calculator tool that demonstrates parameter extraction, tool execution, and response parsing by dispatching to a registered function and returning a string result.

Build an agent loop where the LLM acts as brain, tools provide capabilities, and memory guides observe, act, and repeat for multi-step tasks within a max iterations cap.

Plan and execute with an agent pattern that first creates an explicit step-by-step plan, then executes each step with tools, separating planning from doing for traceability.

Learn the ReAct agent loop—think, act, observe—and compare its transparent reasoning with plan-and-execute, mastering debugging and tool-driven multi-step workflows.

Long term memory provides selective, persistent storage for intelligent agents to remember user preferences and project state across sessions, enabling continuity and personalization.

Engineer stateless LLMs by adding short-term memory for session context and long-term memory for cross-session facts, using memory tools and a JSON file to persist knowledge.

Master task decomposition by breaking big goals into small, verifiable steps aligned to tools and actions with verification checkpoints to catch errors early.

Coordinate multi-agent workflows with structured messages, clear schemas, and explicit state to avoid contradictions. Use typed intents, task IDs, and acceptance criteria to ensure reliable handoffs and aligned goals.

Explore strict tool schemas, observe-act-verify cycles, and parallel tool calls to build reliable agents with memory, guardrails, and structured extraction.

Define and fill a tiny JSON schema to extract structured data from unstructured documents like invoices, then validate and compare zero-shot and few-shot extraction using system prompts and JSON outputs.

Develop a repeatable evaluation mindset that measures every prompt tweak, model swap, and retrieval adjustment with reference-based metrics like Bleu and Rouge, plus LLM-as-judge and rubric scoring.

Compare large language model outputs to a trusted reference or gold answer using deterministic, heuristic, or judge scoring to yield repeatable correctness checks across tasks.

Agent evaluation measures an AI agent's performance across multi-step tasks, tool calls, safety, and cost, using full trajectory analysis rather than final answers alone.

Build a two-layer agent evaluation pipeline that analyzes traces with deterministic checks and an llm rubric judge to diagnose tool selection, parameter accuracy, and answer quality, plus failure patterns.

Wrap up module 9 by outlining practical evaluation techniques: measure regressions, use BLEU and ROUGE for references, deploy judge prompts with rubrics, and automate evals to gate production.

Bridge a demo to a production API using fast api, async concurrency, streaming, caching, retries with backoff and jitter, and observability for cost tracking and diagnosis.

Expose typed, validated ML and LLM endpoints with FastAPI, a modern Python web framework that auto-generates docs, supports sync and async handlers, and centralizes microservices.

Define a stable, structured response format for AI APIs to ensure consistent, versioned, machine readable outputs with clear data or error envelopes, citations, and warnings.

Learn to run multiple LLM calls concurrently with async and asyncio gather using the OpenAI async client, compare sequential versus concurrent performance, and implement per-task error handling for production throughput.

Learn caching strategies for AI apps to reduce repeated LLM calls, embeddings, and retrieved documents with TTL, versioned keys, stale-while-revalidate patterns, spike protection, avoiding pitfalls.

Explore observability for LLM apps using logs, metrics, and traces to reveal what happened, why, and how well, with practical guidance on request IDs and privacy.

Learn how structured logs with correlation IDs enable end-to-end tracing, debugging, and performance monitoring for AI apps. Prioritize privacy by redacting data and logging targeted events only.

Ship production-grade LLM endpoints with FastAPI, async streaming, caching, and retries, guided by robust API contracts and observability, ensuring predictable behavior under load and failure.