Is the Gap Real?
Two simulated models take the same benchmark. Change its size, rerun it on fresh questions, and compare unpaired and paired error bars to see when a lead is more than luck.
From Chapter 16: Evaluation, Reliability, Safety & Interpretability
Try it · toy model
Is the Gap Real?
Know well8 min
Read the concepts