Drop a point on a loss landscape and watch gradient descent, momentum and Adam race to the bottom. Push the learning rate until training diverges; add noise to see stochastic gradient descent.
From Chapter 2: The Math Toolkit
Try it · toy model
Gradient Descent Playground
Read the concepts