3Blue1Brown
Vectors | Chapter 1, Essence of linear algebra
Builds the picture you need for everything else: a vector as an arrow, as a list of numbers, and why both views matter.
Covers: Vectors, vector addition, scaling.
Must know
Lectures & explainers
Watch
A few exceptional videos rather than many mediocre ones — 43 so far, about 29 h 21 min in total. Each is placed where it fits in the curriculum, with a note on why it's worth your time.
Chapter 2
The Math Toolkit
3Blue1Brown
Linear transformations and matrices | Chapter 3, Essence of linear algebra
The single best explanation of why a matrix is a transformation of space — which is exactly what a neural-network layer does.
Covers: Matrices as functions that move vectors; matrix–vector multiplication.
Must know
3Blue1Brown
Dot products and duality | Chapter 9, Essence of linear algebra
Explains why the dot product measures alignment, connecting the arithmetic to the geometry.
Covers: Dot product, projection, the link between dot products and linear maps.
Should know
3Blue1Brown
Matrix multiplication as composition | Chapter 4, Essence of linear algebra
Shows why multiplying matrices means applying one transformation after another — exactly what stacking neural-network layers does.
Covers: Matrix–matrix multiplication as composition of linear maps.
Must know
3Blue1Brown
Eigenvectors and eigenvalues | Chapter 14, Essence of linear algebra
The clearest visual intuition for eigenvectors: directions a transformation only stretches.
Covers: Eigenvectors, eigenvalues, why they matter for understanding a matrix.
Should know
3Blue1Brown
Bayes theorem, the geometry of changing beliefs
Turns Bayes from a formula into a picture of restricting your view to the evidence.
Covers: Prior, likelihood, posterior, and the area picture of Bayes theorem.
Must know
3Blue1Brown
The medical test paradox, and redesigning Bayes' rule
Why a 90%-accurate test can still be wrong most of the time — base rates, made unforgettable.
Covers: Base rates, false positives, the odds form of Bayes rule.
Should know
3Blue1Brown
The essence of calculus
Rebuilds the idea of a derivative from scratch, visually, without assuming you remember school calculus.
Covers: What derivatives and integrals are really about.
Must know
3Blue1Brown
Visualizing the chain rule and product rule | Chapter 4, Essence of calculus
The chain rule is the engine of backpropagation; this makes it feel obvious rather than memorized.
Covers: Chain rule and product rule as nudges propagating through functions.
Must know
3Blue1Brown
Gradient descent, how neural networks learn | Deep Learning Chapter 2
Connects the abstract idea of minimizing a function to how a real network learns to recognise digits.
Covers: Cost functions, gradients in many dimensions, gradient descent.
Must know
StatQuest with Josh Starmer
Gradient Descent, Step-by-Step
Works gradient descent out by hand on a tiny regression problem, one step at a time.
Covers: Loss, derivatives, step size, stochastic gradient descent.
Should know
Aurélien Géron
A Short Introduction to Entropy, Cross-Entropy and KL-Divergence
Ten minutes that tie entropy, cross-entropy and KL divergence together — the three quantities behind language-model training.
Covers: Entropy as average surprise, cross-entropy as loss, KL as the gap between them.
Must know
3Blue1Brown
Solving Wordle using information theory
A playful but rigorous introduction to information and entropy as expected bits of surprise.
Covers: Information in bits, entropy, choosing actions that maximize expected information.
Should know
StatQuest with Josh Starmer
Confidence Intervals, Clearly Explained!!!
Short and concrete — what a confidence interval actually tells you, via bootstrapping.
Covers: Bootstrapping, confidence intervals.
Should know
Chapter 3
Machine Learning
StatQuest with Josh Starmer
Machine Learning Fundamentals: Bias and Variance
Seven minutes on the most important trade-off in machine learning.
Covers: Bias, variance, overfitting, underfitting.
Must know
StatQuest with Josh Starmer
Linear Regression, Clearly Explained!!!
Least squares, R², and what a fitted line actually tells you — from the ground up.
Covers: Fitting a line, residuals, R², p-values.
Should know
StatQuest with Josh Starmer
StatQuest: Logistic Regression
A short, clear picture of how logistic regression turns a line into probabilities.
Covers: The S-shaped curve, classification with probabilities, maximum likelihood.
Must know
StatQuest with Josh Starmer
Decision and Classification Trees, Clearly Explained!!!
How a tree chooses its questions — the most interpretable model in this chapter.
Covers: Splits, impurity (Gini), leaves, building a tree.
Should know
StatQuest with Josh Starmer
StatQuest: Random Forests Part 1 - Building, Using and Evaluating
Why many random trees beat one careful tree.
Covers: Bootstrapping, random feature subsets, voting, out-of-bag error.
Should know
StatQuest with Josh Starmer
StatQuest: Principal Component Analysis (PCA), Step-by-Step
PCA built up geometrically, with no linear algebra assumed beyond the basics.
Covers: Principal components, variance explained, projection.
Should know
StatQuest with Josh Starmer
StatQuest: K-means clustering
The k-means loop, animated.
Covers: Assign, update, repeat; choosing k.
Should know
StatQuest with Josh Starmer
ROC and AUC, Clearly Explained!
How to evaluate a classifier across every possible threshold.
Covers: Confusion matrices, true/false positive rates, ROC curves, AUC.
Should know
StatQuest with Josh Starmer
Machine Learning Fundamentals: Cross Validation
Why you never test on your training data, and how to use limited data well.
Covers: Train/test splits, k-fold cross-validation.
Must know
StatQuest with Josh Starmer
Regularization Part 1: Ridge (L2) Regression
Regularization made concrete: accept a little bias to reduce a lot of variance.
Covers: Ridge penalty, λ, bias–variance, fitting with few data points.
Should know
Chapter 4
Neural Networks
Andrej Karpathy
The spelled-out intro to neural networks and backpropagation: building micrograd
Builds automatic differentiation from nothing; afterwards backpropagation stops feeling like magic.
Covers: Derivatives, the chain rule, computational graphs, backprop, a tiny neural network.
Must know
3Blue1Brown
But what is a neural network? | Deep learning chapter 1
The clearest visual introduction to what a neural network actually computes, layer by layer.
Covers: Neurons, weights, biases, layers, activations — all on digit recognition.
Must know
3Blue1Brown
Backpropagation, intuitively | Deep Learning Chapter 3
Builds intuition for how each training example 'nudges' every weight — before any calculus.
Covers: What backpropagation does, and why stochastic gradient descent uses mini-batches.
Must know
3Blue1Brown
Backpropagation calculus | Deep Learning Chapter 4
The chain rule applied to a network, step by step — the formal version of the previous video.
Covers: Derivatives through layers, the chain rule, sensitivity of the cost to each weight.
Should know
Chapter 5
Vision, Speech & Reinforcement Learning
3Blue1Brown
But what is a convolution?
A visual introduction to discrete convolution that starts with adding dice and ends with image kernels and FFTs.
Covers: Slide, multiply and add; image kernels such as blurs and edge detectors; and why the same operation appears in probability and signal processing.
Must know
StatQuest with Josh Starmer
Neural Networks Part 8: Image Classification with Convolutional Neural Networks (CNNs)
Walks a tiny CNN through a complete example, filter to pooling to fully connected layer, with every number shown.
Covers: Filters, feature maps and max pooling, one step at a time, ending in a small image classifier.
Must know
3Blue1Brown
But what is the Fourier Transform? A visual introduction.
Builds the intuition behind splitting a sound into its frequencies, the operation every spectrogram column performs.
Covers: Winding a signal around a circle, why a frequency present in the signal stands out, and how that separates mixed tones.
Should know
Steve Brunton
Reinforcement Learning: Machine Learning Meets Control Theory
A compact overview of the RL problem, its leading algorithms and applications, from a control-theory perspective.
Covers: Agent, environment, reward, policy and value; Markov decision processes; the credit-assignment problem; and Q-learning.
Should know
Google DeepMind
RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning
The first lecture of the classic UCL course by the lead researcher on AlphaGo. For when you want the full course after this chapter.
Covers: Rewards, the agent–environment loop, states and Markov states, policies, value functions, models, and exploration versus exploitation.
Should know
Chapter 6
Language Before Transformers
Andrej Karpathy
The spelled-out intro to language modeling: building makemore
Builds a character-level bigram model from counts, then trains a one-layer neural network that learns the same table by gradient descent.
Covers: Language modeling, count tables, smoothing, negative log-likelihood, and swapping counting for a trained network.
Should know
Andrej Karpathy
Building makemore Part 2: MLP
Implements the Bengio et al. 2003 neural language model at character level: an embedding lookup, a hidden layer and a softmax over the next character.
Covers: Learned embeddings, a fixed context window, training splits and why sharing vectors beats exact-count tables.
Should know
Chapter 7
Transformers
3Blue1Brown
Transformers, the tech behind LLMs | Deep Learning Chapter 5
A visual tour of a GPT from input text to next-token probabilities — ideal before or right after Chapter 7.
Covers: Tokens, embeddings, the flow of data through a GPT, softmax and temperature.
Must know
3Blue1Brown
Attention in transformers, step-by-step | Deep Learning Chapter 6
Animates exactly what queries, keys and values do. Watch it alongside the Attention Explorer.
Covers: Queries, keys, values, the attention pattern, masking, multi-head attention.
Must know
3Blue1Brown
How might LLMs store facts | Deep Learning Chapter 7
The feed-forward half of a Transformer block gets less attention than attention; this fixes that.
Covers: MLP sublayers, directions in embedding space, superposition (intuition).
Should know
Andrej Karpathy
Let's build GPT: from scratch, in code, spelled out.
The best way to reach IMPLEMENT level on Transformers: write one yourself, line by line, in PyTorch.
Covers: Self-attention, multi-head attention, masking, residuals, layer norm — all built from a bigram baseline up.
Must know
Chapter 8
The Rise of Large Language Models
Andrej Karpathy
Let's build the GPT Tokenizer
Many odd LLM behaviours trace back to tokenization; this shows you why by building a BPE tokenizer.
Covers: Unicode, bytes, byte-pair encoding, GPT-2/GPT-4 tokenizers, special tokens.
Should know
Andrej Karpathy
[1hr Talk] Intro to Large Language Models
A clear one-hour overview of what LLMs are, how they are trained, and where they're going — good orientation for Part III.
Covers: Pretraining, fine-tuning, scaling, tool use, security issues.
Must know
Andrej Karpathy
Deep Dive into LLMs like ChatGPT
A long, general-audience walk through the whole pipeline behind a chat model, from internet text to tokens to pretraining to post-training.
Covers: The full training stack behind a chat model, from internet text and tokens to a pretrained base model and the post-training that turns it into an assistant.
Should know
Andrej Karpathy
Let's reproduce GPT-2 (124M)
Build and train the smallest GPT-2 from scratch in PyTorch, then optimise it. For when you want to implement what this chapter describes.
Covers: The GPT-2 network in code, the engineering that makes training fast, and a full training run using the GPT-2 and GPT-3 papers' hyperparameters.
Frontier