Chain of Thought

Put a result where the next step can use it

Six trays hold seven seedlings each, and half are moved outside. How many remain?

Answering “21” immediately means the model must get from the question to the first token of the answer with $6 \times 7 = 42$ and $42 \div 2$ never written anywhere. Writing “6 × 7 = 42” first changes the situation: the number 42 is now a token in the context, and the next prediction can attend to it like any other input. Each written step is a small, easier prediction conditioned on the previous ones. (This example is written for teaching; it is not a captured model trace.)

A data-pipeline analogy: computing everything in one giant query versus materialising intermediate tables. The intermediate tables cost storage, but each later step becomes simpler and inspectable.

Three ways to get one

Wei and colleagues showed that few-shot prompts containing worked, step-by-step examples improved large models' performance on arithmetic, commonsense and symbolic reasoning tasks in their experiments; the gains appeared mainly in sufficiently large models. Established Kojima and colleagues then found that no examples were needed: adding “Let's think step by step” raised InstructGPT's accuracy on MultiArith from 17.7% to 78.7% and on GSM8K from 10.4% to 40.7%. Established

Keep three mechanisms apart, because they produce similar-looking text:

1. Prompting (worked examples or an instruction) changes the input. Weights are untouched.
2. Supervised fine-tuning on worked solutions changes weights to imitate the solutions.
3. Reinforcement learning changes weights according to a reward on the outcome or the steps, see verifiable rewards.

Today's reasoning models mostly come from the third route, starting from a model that already writes chains of thought.

The probability picture

Write $x$ for the question, $z$ for the intermediate text and $y$ for the answer. The model's overall probability of an answer sums over all the chains it might write:

$$p(y \mid x) = \sum_z p(z \mid x)\, p(y \mid x, z).$$

What it does. One generated response samples (or greedily decodes) a single $z$, then an answer conditioned on it. It does not compute the sum. If the model writes a bad chain, the answer conditions on the bad chain. That is why sampling several chains and voting, self-consistency, approximates the sum better than one chain does.

Costs and failure modes

• Tokens and latency. Every intermediate token is generated sequentially and billed. A chain of 2,000 tokens takes longer than a 20-token answer, whatever the hardware.
• Error propagation. A wrong step early in the chain becomes context for everything after it.
• Padding. Text such as “let me think carefully” adds tokens without adding a partial result. The reward lab shows what happens if length itself is rewarded.
• Parsing. The final answer must be extracted from free text, which is a source of evaluation errors.
• Faithfulness. Turpin and colleagues showed that biasing features in a prompt can change a model's answer without appearing in its chain of thought. Established See reasoning faithfulness.

Mini experiment

Ask any chat model a two-step word problem twice: once with “Answer with only the number” and once with “Show each calculation, then give the number”. Repeat on five problems. Count correct answers and output tokens for each mode. You have measured, on a tiny sample, the trade this page describes: tokens for accuracy. Treat five problems as an anecdote, not a benchmark.