More Features Than Neurons

Train the toy model of superposition live: five features through two hidden neurons. Make the features rarer and watch the model go from keeping two features to packing all five into a pentagon.

From Chapter 16: Evaluation, Reliability, Safety & Interpretability