Your cart is empty.
Keep shopping
LLM Inference in 100 Questions: Serving & Optimization
LLM Inference in 100 Questions: Serving & Optimization
How LLMs are served in production: GPUs, KV cache, batching, quantization, caching and serving architecture
Created byBugra & Şebnem Ayan
Last updated 11/2026
English
ArabicAzerbaijani
What you'll learn
- Explain what happens inside an LLM server from request to response: tokens, prefill, decode and the KV cache
- Estimate GPU memory, latency (TTFT, TPOT), throughput and cost per million tokens for a real deployment
- Apply the key optimizations: continuous batching, PagedAttention, FlashAttention, quantization, prefix caching, speculative decoding
- Design a serving architecture: parallelism, disaggregated prefill/decode, cache-aware routing and autoscaling
- Choose between vLLM, SGLang, TensorRT-LLM, llama.cpp, managed APIs and your own GPUs with confidence
- Diagnose bottlenecks with the roofline model and run an end-to-end optimization study on your own workload
Explore related topics
Course content
10 sections • 100 lectures • 5h 18m total length
Requirements
- Basic familiarity with what LLMs are and how apps call them (e.g. ChatGPT or an API). No GPU or ML research background needed
- Comfort with basic arithmetic; formulas are explained step by step with worked numbers
Description
Every time you send a prompt to a chatbot, a GPU somewhere has to load billions of weights, juggle hundreds of other users and stream your answer back token by token, fast enough to feel instant and cheap enough to make business sense. This course explains how that actually works.
LLM Inference in 100 Questions is a complete, visual tour of large language model serving and optimization, organized as 100 short, focused lessons of about 3 minutes each. Every lesson answers one concrete question, with animated diagrams, worked numbers and a clear takeaway.
What you will cover
How a model generates an answer: tokens, attention, prefill vs. decode, the KV cache and streaming
Hardware and bottlenecks: GPU spec sheets, HBM vs. SRAM, NVLink, memory sizing, arithmetic intensity and the roofline model
Measuring and costing: TTFT, ITL/TPOT, throughput, Little's law, benchmarking and the cost of a million tokens
Batching: static, dynamic and continuous batching, chunked prefill, schedulers and preemption
Attention and memory: MQA, GQA, MLA, FlashAttention, PagedAttention and KV cache quantization
Shrinking the model: FP16, BF16, INT8, FP8, FP4, GPTQ, AWQ, distillation and sparsity
Never doing the same work twice: prefix caching, RadixAttention, semantic caching and speculative decoding (Medusa, EAGLE)
Serving architecture: load balancing, autoscaling, tensor/pipeline/expert parallelism, disaggregated prefill and decode, cache-aware routing
Engines and platforms: vLLM, SGLang, TensorRT-LLM, llama.cpp, managed cloud options, build vs. rent
What's next: multimodal serving, multi-LoRA, edge inference, semantic routing, RL and new accelerators
The course is based on the book "100 Soruda Büyük Dil Modelleri" (Large Language Models in 100 Questions) by Buğra Ayan. The lessons are narrated by the author. English subtitles are included.
No GPU is required to follow along. By the end, you will be able to reason about latency, throughput and cost like an inference engineer, and choose the right optimizations for your own workload.
Who this course is for:
- Software and ML engineers who deploy or integrate LLMs and want them faster, cheaper and more reliable
- Platform, DevOps and MLOps engineers sizing GPUs, planning capacity and running inference in production
- Tech leads, architects and students who want a clear, intuitive picture of how LLM serving really works