Official documentation for vLLM, the high-throughput open-source LLM inference and serving engine powered by PagedAttention, continuous batching, speculative decoding, and quantized kernels.
Originating from UC Berkeley Sky Computing Lab research, vLLM is the leading open-source LLM serving engine. Its documentation explains how PagedAttention eliminates KV-cache fragmentation by managing attention keys and values like OS virtual memory pages, how continuous iteration-level batching maximizes GPU utilization, and how to benchmark quantized (AWQ, GPTQ, FP8) and speculative decoding deployments.
Course status is tracked independently from roadmap topic checkboxes so you can mark external courses without altering your topic mastery percentage.
Phase 8 Module 8.4 Inference Serving & Quantization Reference
100% free open-source Apache-2.0 documentation and benchmarking guides.
Status: Verified (2026-11-09)
Review these foundations before starting so you get the most out of this material:
In Phase 8: Advanced LLM Systems & Research
Study alongside Module 8.4 (Inference & Optimization) and Milestone #13 (Quantized & Optimized Inference Benchmark).
Graduate-level Stanford course walking through every aspect of building language models from scratch: Transformer architecture, tokenization, pretraining data pipelines, GPU systems, parallelism, and scaling laws.
Community-driven advanced research and engineering lecture series covering GPU hardware architecture, CUDA kernels, Triton, FlashAttention, quantization, and PyTorch compiler internals.
Official hands-on tutorials for OpenAI's Triton language: vector addition, fused softmax, tiled matrix multiplication, LayerNorm, and FlashAttention kernels in Python.