Stanford's flagship graduate NLP curriculum covering word vectors (Word2Vec), neural language models, recurrent networks (RNN/LSTM), attention mechanisms, and pretrained Transformers.
Led by Professor Christopher Manning, Stanford CS224N provides world-class lecture slides, detailed reading notes, and coding assignments on distributed word representations, Skip-gram/CBOW embeddings, n-gram and neural language models, vanishing gradients in RNNs/LSTMs, and the transition from sequence bottlenecks to self-attention.
Course status is tracked independently from roadmap topic checkboxes so you can mark external courses without altering your topic mastery percentage.
Phase 4 University Theory & Assignments (Modules 4.3, 4.4 & 4.5)
Lecture slides, comprehensive reading notes, assignment handouts, and archived Stanford YouTube lecture playlists are publicly available.
Status: Verified (2026-11-09)
Review these foundations before starting so you get the most out of this material:
In Phase 4: NLP & Language Modeling
Read the lecture notes and slides for Lectures 1–7 (Word Vectors, Neural Classifiers, RNNs, Language Models, Perplexity, and Vanishing Gradients) alongside Modules 4.3–4.5.
Character, word, and subword tokenization; Byte Pair Encoding (BPE), WordPiece, Unigram, vocabulary sizing, and special control tokens.
Autoregressive next-token prediction, conditional probability factorization, cross-entropy training, perplexity evaluation, and sampling.
Recurrent Neural Networks (RNNs), LSTMs, teacher forcing, seq2seq encoder-decoders, and the information & parallelization bottlenecks that led to Transformers.
Comprehensive open interactive course covering Transformer models, tokenization pipelines, datasets, fine-tuning, and the open-source LLM ecosystem.
A code-first introduction to neural networks covering automatic differentiation, backpropagation, language models, and GPT implementations.
Graduate-level Stanford course walking through every aspect of building language models from scratch: Transformer architecture, tokenization, pretraining data pipelines, GPU systems, parallelism, and scaling laws.