Vanishing and Exploding Gradients

Send a gradient backwards through up to 40 layers and see it shrink to nothing or blow up — and how ReLU and good initialization fix it.

From Chapter 4: Neural Networks