Spend a Compute Budget

Split a fixed number of FLOPs between model size and training tokens with a published scaling law, place real models on the map, and see how serving costs change the answer.

From Chapter 9: How an LLM Is Actually Built