NVIDIA Deep Learning Institute (DLI) self-paced training catalog featuring both free introductory courses and paid hands-on GPU cluster labs on CUDA, accelerated computing, and LLM deployment.
The NVIDIA Deep Learning Institute (DLI) self-paced catalog offers browser-accessible technical modules on accelerated computing, CUDA Python/C++, multi-GPU scaling, and TensorRT-LLM inference. Important Access Note: This is a mixed free and paid training catalog—not every course in the catalog is free. Filter by 'Free' inside the NVIDIA catalog to access complimentary self-paced modules.
Course status is tracked independently from roadmap topic checkboxes so you can mark external courses without altering your topic mastery percentage.
Phase 8 Supplementary Mixed Catalog (Modules 8.1, 8.2 & 8.4)
Mixed Free & Paid Catalog: Contains complimentary introductory self-paced courses alongside paid cloud-GPU lab courses. Use the catalog's price filter to select free modules.
Certificate Note: Paid DLI courses and certification exams carry a fee; free introductory modules do not require payment.
Status: Verified (2026-11-09)
Review these foundations before starting so you get the most out of this material:
In Phase 8: Advanced LLM Systems & Research
Optional supplementary catalog for Phase 8. Note: This is a mixed free/paid catalog—filter for free self-paced courses if you want zero-cost modules on CUDA and accelerated AI.
GPU hardware architecture, CUDA execution hierarchy, HBM vs. SRAM memory bandwidth, kernel fusion, FP16/BF16, activation checkpointing, FlashAttention, Triton, and PyTorch Profiler.
Scaling training across multiple GPUs and nodes: DDP, FSDP (ZeRO-1/2/3), Tensor Parallelism, Pipeline Parallelism, collective communication primitives, and fault tolerance.
Post-training quantization (INT8, INT4, AWQ, GGUF), KV cache management, continuous batching, PagedAttention, speculative decoding, serving APIs, observability, and cost optimization.
Official NVIDIA Developer & Educator teaching kits, university course materials, and technical resources on parallel computing and CUDA GPU programming.
Community-driven advanced research and engineering lecture series covering GPU hardware architecture, CUDA kernels, Triton, FlashAttention, quantization, and PyTorch compiler internals.
Official PyTorch documentation and step-by-step tutorials for DistributedDataParallel (DDP), Fully Sharded Data Parallel (FSDP2), Tensor Parallelism, and multi-GPU collective communication.