Five parts, read in order. The road shows reading order, not a single path through history. Point at a part to see what it covers.
Most explanations of AI start with the newest model and work backwards. This one starts in 1950 and walks forward. Each idea arrives because the one before it ran into a wall: rules couldn't keep up with the world, so machines learned from examples; reading one word at a time was too slow, so attention let every word see every other.
It's written for engineers who use AI every day and want the theory underneath to click. Every paper and video is checked against its source, and claims about fast-moving research say how settled they are.
Each star is one concept page, placed by a real embedding model: pages it reads as similar sit close together. Point at a star to see its nearest neighbours and what it builds on, click to fly in, or rearrange the stars along the road you’ll walk.
16 chapters are open now. Each one starts from a problem and ends where the next one begins. To feel how far the road goes, take the long descent: 75 years of training compute on one scale.
Type anything. This is the real tokenizer from GPT-2 and GPT-4, splitting your text the way the model receives it. Every chapter has experiments like this one.
Learn those numbers from text, and words used in similar places end up close together. Meaning becomes distance.
Illustration of an embedding space. Real ones have hundreds of dimensions.Read the concept →
04
Attention
To understand “it”, score every other word with a dot product, turn the scores into weights, and mix. Here “it” gives 54% of its attention to “cat”.
What “it” attends to, using the hand-set toy weights from the Attention from Scratch lab.Read the concept →
05
Self-attention
Do that for every word at once and you get a table of weights. That table is computed inside every layer of a large language model.
Every word at once: rows ask, columns answer. Weights in %, same toy model.Read the concept →
Toy 2-D vectors, so they can be drawn. Real models use hundreds of dimensions.
The road so far
The questions that moved AI forward.
Each answer raised the next question, from “Can machines think?” in 1950 to models that think before they answer. Watch them unfold as a short animated story, or follow every step on the timeline.
An animated tour of these moments, about four minutes. It starts only when you press play.
Your progress, kept
Pick up exactly where you stopped.
Mark as you go. Every concept can be set to Learning, Understood or Revisit; chapters, papers, videos and experiments can be ticked off.
Sign in to keep it. With Google or an email link, your progress is saved to your account and follows you to any device. We store only your email and your progress.
Without an account, it's temporary. You can mark things without signing in, but they're kept only in this browser: clearing your browsing data, switching devices or, in Safari, seven days without a visit erases them.
Ch 1
Ch 2
Ch 3
Ch 4
Ch 5
Ch 6
Ch 7
Ch 8
Ch 9
Ch 10
Ch 11
Ch 12
Ch 13
Ch 14
Ch 15
Ch 16
219 concepts, one block each. Mark a concept Understood and its block fills in.