Rigorous introduction to discrete and continuous probability distributions, conditional probability, Bayes' theorem, expectation, variance, and maximum likelihood estimation.
Language models are probabilistic next-token predictors trained via Maximum Likelihood Estimation (equivalent to minimizing cross-entropy loss). MIT 18.05 provides crystal-clear class reading notes, studio worksheets, and problem sets covering random variables, expectation, variance, Bayes' rule, and MLE.
Course status is tracked independently from roadmap topic checkboxes so you can mark external courses without altering your topic mastery percentage.
Phase 1 Module 1.3 (Probability, Statistics & Information Theory)
Complete reading notes, studio exercises, exams, and solutions are freely available on MIT OCW.
Status: Verified (2026-11-09)
Review these foundations before starting so you get the most out of this material:
In Phase 1: Mathematics for AI
Study the class reading notes on Random Variables, Expectation/Variance, Conditional Probability, and Maximum Likelihood Estimation during Module 1.3.
Mathematical foundations of machine learning, including linear algebra, vector calculus, probability, and optimization.
A university-level linear algebra course covering vector spaces, matrices, linear systems, eigenvalues, and related topics.