Will It Fit?
Add up the memory a GPU needs to train a real model: weights, gradients, optimizer states and activations. Then shard and recompute until it fits in 80 GB.
From Chapter 9: How an LLM Is Actually Built
Try it
Will It Fit?
Know well8 min
Add up the memory a GPU needs to train a real model: weights, gradients, optimizer states and activations. Then shard and recompute until it fits in 80 GB.
From Chapter 9: How an LLM Is Actually Built
Try it
Will It Fit?