CS50's Introduction to Programming with Python
A comprehensive introduction to Python programming, covering functions, variables, control flow, exceptions, libraries, unit testing, file I/O, and object-oriented programming.
Explore carefully curated courses, tutorials, lectures, documentation, and research resources from leading universities, educators, and open-source communities — organized around your AI and LLM learning journey.
Canonical Resources
51 Phase Recommendations
Universities & Labs
“Start Here” Picks
How to Use This Library: You do not need to complete every resource listed here. Each phase highlights a primary Start Here resource alongside optional lectures, technical documentation, and reference codebases so you can tailor your study depth.
Bookmark courses and track your resource status (Not Started, In Progress, or Completed) separately from your roadmap topic checklist.
Follow this focused spine if you want one clear starting course per phase without getting overwhelmed by supplementary options.
Harvard CS50 Python
3Blue1Brown + MIT 18.06SC Linear Algebra
Google Machine Learning Crash Course
Karpathy Neural Networks: Zero to Hero
Hugging Face LLM Course
Karpathy Zero to Hero — GPT
Stanford CS336: Language Modeling
Hugging Face smol-course
Stanford CS336 + GPU MODE Lectures
Develop the Python, algorithmic, and numerical computing fluency required to implement machine learning algorithms from first principles.
A comprehensive introduction to Python programming, covering functions, variables, control flow, exceptions, libraries, unit testing, file I/O, and object-oriented programming.
Beginner-friendly Python instruction covering programming fundamentals, data structures, files, web data, and databases.
Learn numerical arrays, broadcasting, vectorized computation, matrix operations, and scientific Python fundamentals.
Learn to load, inspect, filter, clean, and manipulate structured datasets.
Learn scientific plotting, mathematical visualization, and data visualization in Python.
Master the linear algebra, multivariable calculus, probability theory, and numerical optimization that govern how neural networks represent and learn.
Visual explanations of vectors, transformations, matrix multiplication, eigenvectors, and the intuition behind linear algebra.
A university-level linear algebra course covering vector spaces, matrices, linear systems, eigenvalues, and related topics.
Intuitive visual lessons explaining derivatives, integrals, and fundamental calculus concepts.
Comprehensive MIT OpenCourseWare Scholar course covering differentiation, the chain rule, optimization of single-variable functions, and integration.
University course on vectors, matrices, partial derivatives, gradient vectors, directional derivatives, and multivariable optimization.
Rigorous introduction to discrete and continuous probability distributions, conditional probability, Bayes' theorem, expectation, variance, and maximum likelihood estimation.
Mathematical foundations of machine learning, including linear algebra, vector calculus, probability, and optimization.
Understand how algorithms learn patterns from data, optimize loss functions, avoid overfitting and data leakage, and generalize to unseen examples.
Google's practical, interactive introduction to machine learning fundamentals, linear and logistic regression, gradient descent, generalization, regularization, and neural networks.
University course exploring the foundational concepts and algorithms of modern artificial intelligence, including search, optimization, machine learning, neural networks, and natural language processing.
Hands-on interactive micro-course covering decision trees, model validation, underfitting vs. overfitting, and random forests.
Practical interactive course covering missing values, categorical variables, pipelines, cross-validation, gradient boosting (XGBoost), and data leakage.
Official scikit-learn getting started tutorials and user guide for classical supervised/unsupervised learning, preprocessing, cross-validation, and evaluation metrics.
Free interactive textbook combining mathematical explanations, code, and practical deep-learning concepts.
Implement multi-layer neural networks, reverse-mode automatic differentiation, and PyTorch training pipelines from the ground up.
A code-first introduction to neural networks covering automatic differentiation, backpropagation, language models, and GPT implementations.
Top-down, code-first course teaching practical deep learning, neural network training dynamics, transfer learning, and PyTorch foundations.
Official 'Learn the Basics' tutorial series covering PyTorch Tensors, Datasets/DataLoaders, `nn.Module`, `torch.autograd`, optimization loops, and model checkpointing.
Free interactive textbook combining mathematical explanations, code, and practical deep-learning concepts.
Stanford's foundational deep learning course materials covering linear classifiers, optimization, backpropagation computational graphs, neural network training dynamics, and representation learning.
Understand how raw human text is normalized, tokenized into subword integer IDs, embedded into dense vectors, and modeled autoregressively.
Comprehensive open interactive course covering Transformer models, tokenization pipelines, datasets, fine-tuning, and the open-source LLM ecosystem.
Stanford's flagship graduate NLP curriculum covering word vectors (Word2Vec), neural language models, recurrent networks (RNN/LSTM), attention mechanisms, and pretrained Transformers.
Canonical Series: Neural Networks: Zero to Hero
A code-first introduction to neural networks covering automatic differentiation, backpropagation, language models, and GPT implementations.
Step-by-step PyTorch tutorial building a character-level Recurrent Neural Network from scratch without high-level RNN wrappers.
Dedicated technical module diving deep into subword tokenization algorithms: Byte-Pair Encoding (BPE), WordPiece, Unigram, normalization, pre-tokenization, and building tokenizers from scratch.
Understand and implement every component of modern GPT-style decoder-only Transformer architectures from first principles in PyTorch.
Canonical Series: Neural Networks: Zero to Hero
A code-first introduction to neural networks covering automatic differentiation, backpropagation, language models, and GPT implementations.
Graduate-level Stanford course walking through every aspect of building language models from scratch: Transformer architecture, tokenization, pretraining data pipelines, GPU systems, parallelism, and scaling laws.
Line-by-line PyTorch implementation and mathematical walkthrough of the original 'Attention Is All You Need' Transformer paper.
The simplest, fastest, and most readable open-source repository for training and fine-tuning medium-sized GPT language models (~300 lines of model code and ~300 lines of training loop).
The landmark 2017 research paper by Vaswani et al. that introduced the Transformer architecture based entirely on self-attention mechanisms, replacing recurrence and convolutions.
Engineer end-to-end dataset pipelines, pretraining loops, evaluation suites, and reproducible packaging to train your own GPT-style language model from random weights.
Graduate-level Stanford course walking through every aspect of building language models from scratch: Transformer architecture, tokenization, pretraining data pipelines, GPU systems, parallelism, and scaling laws.
Standalone 2-hour hands-on video lecture building and training a generative pretrained Transformer (GPT) from an empty Python script.
Canonical Series: nanoGPT
The simplest, fastest, and most readable open-source repository for training and fine-tuning medium-sized GPT language models (~300 lines of model code and ~300 lines of training loop).
Dedicated course chapter on loading, slicing, filtering, deduplicating, streaming, and memory-mapping massive text datasets using Apache Arrow and Hugging Face Datasets.
Comprehensive engineering course covering the end-to-end lifecycle of production deep learning and LLM systems: data management, experiment tracking, troubleshooting, testing, deployment, and continual learning.
Transform pretrained base language models into helpful, safe, instruction-following assistants using SFT, LoRA/QLoRA, DPO, RLHF, and reasoning RL.
Practical hands-on course focused on post-training small and medium open-weight language models: chat templates, Supervised Fine-Tuning (SFT), LoRA/QLoRA parameter-efficient adaptation, preference alignment (DPO/ORPO), and evaluation.
Foundational course module covering dataset preprocessing, dynamic padding, the Trainer API, custom PyTorch fine-tuning loops, and learning rate schedules for adapting pretrained Transformers.
Official documentation and technical guides for TRL (Transformer Reinforcement Learning), covering `SFTTrainer`, `RewardTrainer`, `PPOTrainer`, `DPOTrainer`, and `GRPOTrainer`.
Production recipes, hyperparameter configurations, and open-source training scripts for aligning language models (such as Zephyr and SmolLM-Instruct) using SFT, LoRA/QLoRA, and DPO.
Free interactive course teaching foundational and advanced Reinforcement Learning: Markov Decision Processes, Policy Gradients, Actor-Critic methods, and Proximal Policy Optimization (PPO).
Master GPU kernel architecture, distributed multi-GPU training, Mixture-of-Experts and GQA architectures, high-throughput inference serving, and rigorous AI research methodology.
Canonical Series: CS336 — Language Modeling from Scratch
Graduate-level Stanford course walking through every aspect of building language models from scratch: Transformer architecture, tokenization, pretraining data pipelines, GPU systems, parallelism, and scaling laws.
Community-driven advanced research and engineering lecture series covering GPU hardware architecture, CUDA kernels, Triton, FlashAttention, quantization, and PyTorch compiler internals.
Official NVIDIA Developer & Educator teaching kits, university course materials, and technical resources on parallel computing and CUDA GPU programming.
NVIDIA Deep Learning Institute (DLI) self-paced training catalog featuring both free introductory courses and paid hands-on GPU cluster labs on CUDA, accelerated computing, and LLM deployment.
Official PyTorch documentation and step-by-step tutorials for DistributedDataParallel (DDP), Fully Sharded Data Parallel (FSDP2), Tensor Parallelism, and multi-GPU collective communication.
Comprehensive engineering course covering the end-to-end lifecycle of production deep learning and LLM systems: data management, experiment tracking, troubleshooting, testing, deployment, and continual learning.
Official hands-on tutorials for OpenAI's Triton language: vector addition, fused softmax, tiled matrix multiplication, LayerNorm, and FlashAttention kernels in Python.
Official documentation for vLLM, the high-throughput open-source LLM inference and serving engine powered by PagedAttention, continuous batching, speculative decoding, and quantized kernels.