Adjust a model's scores for the next word and watch cross-entropy loss and perplexity respond — the exact quantity language models are trained to minimize.
From Chapter 2: The Math Toolkit
Try it · toy model
Confident and Wrong
Read the concepts