Circuits and Mechanistic Interpretability

The residual stream as a bus

Elhage and colleagues described a Transformer's residual stream as a communication channel: each attention head and MLP reads from it and adds its output back, so the components act as independent, additive writers. Established For a data engineer: a shared message bus where every stage reads the current record, appends a field and passes it on. A circuit is a chain of such stages that implements one behaviour.

Induction heads

The cleanest example. In small attention-only Transformers, the same authors found that two-layer models compose heads into "induction heads", which only appear with at least two attention layers and explain in-context learning in those models. Established

It takes two heads:

1. A previous-token head in the first layer writes, at each position, "the token before me was …".
2. An induction head in the second layer, at the current token [A], looks for earlier positions whose previous token was [A], attends to them, and copies what is there: [B].

So "Mr Dursley … Mr" predicts "Dursley". Olsson and colleagues linked induction heads to in-context learning in larger models with several lines of evidence, causal in small models and correlational in larger ones. Established

Activation patching

Finding components is half the job; showing they matter is the other. Meng and colleagues ran the model on a prompt such as "The Space Needle is in downtown" and on a corrupted copy with the subject obscured, then restored individual internal activations from the clean run; restoring mid-layer MLP activations at the subject's last token brought the right answer back. Established They then edited single facts by a rank-one update to one MLP's weights. Established Hase and colleagues later found that where causal tracing localized a fact did not predict which layer was best to edit. Established Localizing and controlling are different questions.

A circuit in the wild

Wang and colleagues explained how GPT-2 small completes "When Mary and John went to the store, John gave a drink to" with "Mary": 26 attention heads in 7 classes, found with causal interventions and evaluated for faithfulness, completeness and minimality, criteria that also exposed remaining gaps. Established In a small Transformer trained on modular addition, Nanda and colleagues reverse-engineered the complete algorithm, which uses discrete Fourier transforms and trigonometric identities. Established

Tiny example

Sequence: "A7 B3 … A7". At the last token, the induction head's query is "find positions whose previous token was A7". Only the position of "B3" qualifies (its previous token is A7), so the head attends there and copies "B3" into the stream, raising its logit. Patch in the first layer's output from a sequence where the earlier pair was "A7 C9", and the prediction changes to "C9": evidence the head is copying, not guessing.

Mini experiment

Write a 12-token sequence with one repeated pair, such as "red 4 blue 9 green 2 red". By hand, mark which earlier position an induction head should attend to at the last token and what it predicts. Then write one where the pattern is ambiguous (two earlier "red"s followed by different tokens). What should the head do, and what would you measure to find out?