Sparse Autoencoders and Dictionary Learning

Undoing superposition

If a layer's activation $x$ is a sum of a few active features out of many, then finding those features is a dictionary learning problem: learn a large set of directions $d_1, \dots, d_F$ (with $F$ much bigger than the layer width) such that every activation is approximately a sparse combination of them.

A sparse autoencoder does this with a neural network:

$$f = \mathrm{ReLU}(W_{\text{enc}} x + b_{\text{enc}}), \qquad \hat x = b_{\text{dec}} + \sum_i f_i\, d_i, \qquad \mathcal{L} = \lVert x - \hat x\rVert^2 + \lambda \sum_i |f_i| .$$

The L1 penalty pushes most $f_i$ to zero for any given input, so each input is explained by a few features. To label a feature, look at the inputs that activate it most.

From a small model to a production one

Towards Monosemanticity trained sparse autoencoders on 8 billion activations of the 512-neuron MLP layer of a one-layer Transformer, with dictionaries from 512 to 131,072 features, and studied a 4,096-feature run in detail; features such as Arabic script, DNA sequences, base64 and Hebrew were much more interpretable than the neurons. Established Cunningham and colleagues independently found sparse-autoencoder features more interpretable than other decompositions by automated measures. Established Scaling Monosemanticity trained dictionaries of about 1M, 4M and 34M features on the middle-layer residual stream of Claude 3 Sonnet, finding abstract, multilingual and multimodal features, including safety-relevant ones such as features related to deception and bias. Established

Testing that features are used

A feature that fires on Golden Gate Bridge text could be a correlate the model ignores. Clamping that feature to 10 times its maximum activation during the forward pass made Claude 3 Sonnet start to identify itself as the Golden Gate Bridge. Established Intervening and watching behaviour change is the same logic as causal tests of probes and circuits.

Limits

Dictionaries do not reconstruct activations perfectly, so some of the model's computation lives in the error term; many concepts have no clean feature at a given dictionary size; features split into finer ones as the dictionary grows; and their labels are human interpretations of activation examples. Whether sparse autoencoders find the model's "true" features is an open question. Active research

Tiny example

Take the pentagon from the superposition lab: two hidden numbers, five feature directions. Given a hidden vector that is exactly feature 3's direction, any two directions could reconstruct it, but the sparsest explanation is "feature 3 alone". That preference for the sparsest explanation, at scale, is what the L1 penalty encodes.

Mini experiment

Open the interactive feature browser linked from Towards Monosemanticity and pick three features at random. For each, write your own label from the top activating examples before reading the authors'. How often do you agree, and what would you need to see to trust a label?