GenAI Engineer Interview Prep: RAG, Embeddings, LLM Agents

Ever wondered what actually happens inside an LLM? This is the beginner tier of a three-part deep dive into Transformer and LLM architecture internals built from zero, no prior deep-learning background assumed. We start with what a token even is, build embeddings, derive self-attention(softmax(QKᵀ/√d)V) from the 2017 "Attention Is All You Need" paper, then assemble a full Transformer block and the KV cache that makes generation fast. Every episode follows the same rhythm: background with real papers, the mechanism derived, runnable PyTorch, a non-obvious insight, and an animated diagram.

**What you'll learn**

- Tokens and embeddings: how text becomes vectors

- Language modeling as next-token prediction (and why it's compression)

- Attention, from intuition to the exact formula

- Multi-head attention and the full Transformer block

- The KV cache and positional encodings

The clean Transformer works but it's slow and memory-hungry at scale. This intermediate tier covers the tricks that make real LLMs practical: grouped-query attention, rotary positions (RoPE), FlashAttention, Mixture-of-Experts, the KV cache memory math, PagedAttention, and sliding-window long context. The unifying theme: at inference, attention is memory-bound, not compute-bound. Every technique here either shrinks the bytes you move or moves them less.

**What you'll learn**

- Why decoding is memory-bandwidth bound (the roofline view)

- MHA vs MQA vs GQA and how each sizes the KV cache

- RoPE: encoding position by rotation

- FlashAttention: exact attention, IO-aware tiling

- Mixture-of-Experts and load balancing

- KV cache math and PagedAttention (vLLM)

- Sliding-window / local attention

The advanced tier pushes every idea to its research-grade limit: FlashAttention-2/3 on Hopper GPUs, RoPE scaling (Position Interpolation, NTK-aware, YaRN), Mixture-of-Experts routing internals, KV cache compression with Multi-head Latent Attention, sub-quadratic models like Mamba, and streaming/distributed context via attention sinks and Ring Attention. Each topic is grounded in a named 2023–2024 paper, with derivations, PyTorch, and diagrams. This is the material that separates a strong candidate from an expert.

**What you'll learn**

- FlashAttention-2/3: parallelism, warp-specialization, FP8 - RoPE scaling for 100k+ contexts (PI, NTK-aware, YaRN) - MoE routing: load balancing, expert parallelism, fine-grained experts

- KV compression: Multi-head Latent Attention, quantization, eviction

- Sub-quadratic attention and Mamba state-space models

- Streaming (attention sinks) and Ring Attention

Retrieval-Augmented Generation (RAG) is one of the most common topics in AI / ML engineer interviews and the questions go deeper than "what is RAG." In this beginner-level deep dive, we break down the 7 foundational RAG interview questions, building each answer from intuition to the exact nuance interviewers are listening for. Whether you're prepping for an AI Engineer, ML Engineer, or Applied Scientist role, this is your foundation before we move into the intermediate and architect levels.

The fundamentals get you in the door. These 10 intermediate questions are where AI / ML engineer interviews actually separate candidates. This is a deep dive into the retrieval techniques, evaluation discipline, and production gotchas that decide whether a RAG system works in the real world. Every question is built in six layers: the background, the solution, an animated worked example, a technical deep-dive, the state-of-the-art frontier, and exactly what the interviewer is really testing.

What you'll learn: How Reciprocal Rank Fusion (RRF) actually merges keyword and vector search, with a worked example Why a cross-encoder re-ranker is more accurate but can't run over your whole corpus How to compute faithfulness, answer relevance, and context precision/recall The "lost in the middle" U-curve and how to mitigate it Frontier methods: SPLADE, RankGPT, GraphRAG, Self-RAG, late chunking, contextual retrieval, Matryoshka embeddings, and more Who it's for: anyone prepping for an AI Engineer, ML Engineer, Applied Scientist, or LLM/GenAI role — or any engineer building production RAG.

Anyone can build a RAG demo. Designing one that survives 500 million documents, strict latency budgets, multi-tenancy, and high-stakes domains is what separates senior engineers — and it's exactly what system-design interviews probe. This is the architect level: 10 questions on designing, scaling, and operating production RAG. Every question is built in six layers — the background, the solution, an animated worked example, a technical deep-dive, the state-of-the-art frontier, and what the interviewer is really testing.

Choosing an ANN index for your recall, latency, and memory budget — and the knobs that move it (M, ef-search, n-probe, PQ) Sharding and a real latency budget to hit a sub-200ms p99 tail Keeping an index fresh: incremental upserts, tombstones, blue-green swaps Quality observability that catches silent retrieval decay before users do When agentic RAG and GraphRAG actually justify their cost Multi-tenant isolation without destroying recall Why models hallucinate even with correct context (knowledge conflict & parametric override) — and how to fix it A concrete plan to cut serving cost ~70% Citation grounding, abstention, and calibration for safety-critical RAG Hierarchical chunking and retrieval for contracts and API docs Frontier methods covered: ScaNN, DiskANN, RaBitQ, Self-RAG, GraphRAG, conformal prediction, RAPTOR, Matryoshka embeddings, and more. Who it's for: anyone interviewing for Senior / Staff AI Engineer, ML Engineer, or LLM system-design roles — or building production RAG today.

Preparing for an AI/ML engineer interview? Fine-tuning large language models is one of the topics that separates people who have *read* about LLMs from people who have actually *trained* them. In this complete beginner masterclass, we work through the 15 fundamental fine-tuning questions interviewers really use — with the background, the intuition, and the "out-loud" answer a strong candidate would give. No fluff. Every answer is built from first principles with clear visuals, and includes the common traps that quietly sink real training runs.

WHAT YOU'LL LEARN - The difference between pretraining, SFT, instruction tuning, and alignment — and why order matters - When to fine-tune vs. when to just prompt (or use retrieval) - Catastrophic forgetting, the epoch myth, and how to read a loss curve - Loss masking, chat templates, and how a single training example is really structured - Full fine-tuning vs. parameter-efficient fine-tuning (LoRA) — and what "efficient" actually saves - How to choose learning rate, epochs, and batch size — and how decoding settings affect evaluation

This is Part 2 of the LLM fine-tuning interview series — the intermediate tier, where "I know what LoRA is" turns into "I can derive why it works." We go past definitions into mechanisms, trade-offs, and the memory math you're expected to reason through out loud. Every answer is built from first principles with clear visuals, named papers, and concrete worked examples (real numbers, real GPU budgets). If you're interviewing for an AI/ML engineer role — or you fine-tune models and want to actually understand the machinery —this is the deep dive.

WHAT YOU'LL LEARN - The low-rank hypothesis behind LoRA, what rank r and alpha really do, and where to attach adapters - QLoRA end to end: NF4, double quantization, paged optimizers — and the accuracy/compute cost - RLHF vs DPO: the implicit reward, the KL/reference "leash," and how reward hacking happens - Mixed precision (fp16 vs bf16 vs fp8), gradient checkpointing, and the 16-bytes/param memory rule - Sequence packing (and its leakage trap), gradient accumulation, and evaluation beyond loss

This is Part 3 — the architect tier — of the LLM fine-tuning interview series. Beginner built the mental model, intermediate taught the mechanisms; this final series is about the decisions a senior or staff engineer is hired to get right: which approach to use, how to scale it across many GPUs, how to serve it, and how to keep it safe. Every answer is built from first principles with clear visuals, named systems and papers, and concrete worked examples — real GPU math, real trade-offs. If you're interviewing for a senior AI/ML role, or you own LLM systems in production, this is the deep dive. WHAT YOU'LL LEARN When to finetune vs RAG vs tools — and why a small model usually can't just "replace GPT-4" Distributed training: DDP vs ZeRO-1/2/3 vs FSDP, and tensor vs pipeline parallelism Mixture-of-Experts finetuning, and extending context length with Position Interpolation / YaRN Serving 50 finetunes with multi-LoRA, plus quantization-for-serving pitfalls Data flywheels, safe rollout, reproducibility, model merging, the alignment tax, and finetuning's effect on safety

The first level of my LLM Agents deep dive. If you understand language models but "agents" still feels fuzzy, this is the video that makes it click. We go from the precise definition of an agent all the way to building one three different ways — framework-agnostic, OpenAI Agents SDK, and LangGraph. No hand-waving: every question gets background, a worked example, pseudocode, and a clear visual. Built for people who want to actually understand agentic systems, not just collect buzzwords.

What you'll learn:

- What an LLM agent is, vs a plain LLM call and vs RAG

- The five core components every agent shares

- How tool / function calling actually works

- The ReAct loop, agent memory, and how to keep agents from running forever

- Building a weather + calculator agent in three frameworks

Level 2 of my LLM Agents deep dive. The beginner level explained what an agent is — this one is about designing agents you can actually trust, afford, and measure. We cover the patterns and when to use each, handling hundreds of tools with the Model Context Protocol (MCP), real memory design, orchestration frameworks, and the topic that separates demos from products: evaluation. Plus three hands-on scenarios — agentic RAG, recovering from a flaky API, and taming a slow, expensive agent. Code shown in framework-agnostic pseudocode, the OpenAI Agents SDK, and LangGraph.

What you'll learn:

- ReAct vs Plan-and-Execute vs Reflexion — a real decision framework

- Planning and task decomposition

- Tools at scale + MCP (Host/Client/Server)

- Reliable tool calling: structured outputs, validation, self-repair

- Designing memory; orchestration frameworks (loop vs state graph)

- Agentic RAG, error recovery, cost/latency, and agent evaluation

The final level of my LLM Agents deep dive — the senior, production-grade material. Here we treat agents as what they really are: distributed systems with a stochastic core. We cover multi-agent architectures and how agents communicate, controlling context at scale, long-horizon agents that run for hours, production reliability for systems that move money, deep cost/latency optimization, the security threat model (prompt injection and the "lethal trifecta"), multi-agent coordination failures, production observability and evaluation, and finally training agentic behavior with SFT and reinforcement learning. Code in framework-agnostic pseudocode, the OpenAI Agents SDK, and LangGraph.

What you'll learn:

- Multi-agent topologies and agent-to-agent communication

- Context engineering, compaction, and sub-agent isolation

- Long-horizon agents: durable execution, checkpointing, anti-drift

- Production reliability: state machines, HITL, rollback

- Cost/latency: routing, caching, parallelism, speculation

- Security: prompt injection, least privilege, MCP supply chain - Multi-agent failure & recovery; eval and observability; RL for agents

How do you actually prove a large language model is any good? It's the part of ML interviews that quietly separates strong candidates from the rest and it's rarely taught well.

This is the beginner tier of a 3-part series. We answer the ten LLM-evaluation questions interviewers keep asking, in plain language, each one grounded in the real papers, benchmarks, and tools and each with a concrete worked example so the idea actually sticks.

By the end you'll be able to explain, clearly and correctly:

Core concepts

• Why an LLM can grade another LLM  and the three biases (position, verbosity, self-preference) that wreck it

• What benchmark contamination is, and why a 90% score can mean nothing

• How to detect hallucinations automatically (SelfCheckGPT, FActScore)

• What you actually measure in a RAG pipeline and an agent (the four RAGAS metrics, pass^k)

• Offline vs. online evaluation and why you need both

Practical & recent

• How to build eval datasets and rubrics that hold up (and what inter-annotator agreement tells you)

• How to evaluate reasoning models and test-time compute (pass@k vs. cons@k, CoT faithfulness)

• How to wire evals into CI like unit tests (DeepEval, Promptfoo, LangSmith, Braintrust)

• How to measure safety: jailbreaks vs. over-refusal (HarmBench, StrongREJECT)

• How to report error bars and which benchmarks still matter in 2026 (HLE, SWE-bench Verified, RULER)

Who it's for: ML/AI engineers, data scientists, and anyone prepping for an LLM or applied-science interview who wants the clean, correct version of each answer.

Definitions get you a callback. Numbers get you the offer.

This is the intermediate tier. We take the same ten LLM-evaluation questions and attach a worked calculation to each the kind of quantitative reasoning a senior interviewer is actually listening for. Every question ends in a number you can defend.

What you'll be able to compute and explain:

• Validating an LLM judge with Cohen's kappa — and reading it on the Landis–Koch scale

• Quantifying contamination: overlap rates, the public-vs-fresh accuracy gap, and Min-K% probability

• Scoring a hallucination detector with precision/recall/F1 and choosing the threshold by cost

• Computing RAGAS metrics per query — and why pass^k decays so fast for agents

• Sizing an A/B test (and why a 2% lift needs ~10,000 sessions per arm)

• Measuring annotation quality with Krippendorff's alpha, plus stratified coverage

• Trading accuracy against compute for reasoning models (cons@k, pass@1 honesty)

• Designing a CI gate that doesn't flake (temp 0, pass-rate thresholds, tolerance)

• Turning the safety trade-off into an operating curve with StrongREJECT

• Paired, clustered, and bootstrap error bars  the free variance reduction most people miss

Who it's for: engineers who know the concepts and want to sound credible under follow-up questions to put a kappa, an F1, a pass^k, or a sample size on every claim.

This is the whiteboard conversation for staff and principal interviews. Not "which metric," but how the entire evaluation system is built, instrumented, and made cheap enough to run continuously.

The advanced tier takes all ten LLM-evaluation questions to system-design depth. Every question ends in a concrete artifact  a schema, a formula, a named failure pattern, or a piece of pseudocode you could hand to a team.

What you'll be able to design and defend:

• A layered, calibrated judge ensemble with ECE calibration and why optimizing one fixed judge is reward hacking

• A leak-resistant benchmark with canaries and perturbation twins, scored by a ConStat-style z-statistic

• A claim-graph verification pipeline (SUPPORTED / REFUTED / NEI) with a cost-saving cascade

• Agent trace-span schemas, process-vs-outcome scoring, and a real eval cost model

• Variance reduction (CUPED), interleaving, always-valid inference, and safe ramps (SRM, novelty effects)

• A versioned annotation pipeline with adjudication and drift monitoring

• Compute-matched Pareto evaluation and a quantified CoT-faithfulness test

• An eval-platform architecture with statistically-aware, paired-delta gating

• A co-evolving attacker–defender red-team loop with severity-weighted risk

• Matching the error model to the data  paired, clustered, bootstrap, and the delta method for Elo

Who it's for: senior/staff ML engineers and applied scientists who need to show systems thinking  evaluation as a closed loop, not a single number.