If you have ever added “show me the steps” to a request and got a better answer, you were unknowingly using the result of this paper. The question it answers is simple: does asking the model to write out its reasoning before the answer make it better? In 2022 a group at Google (Jason Wei and colleagues) measured that yes, it does, but only from fairly large models up. The name they gave the technique, chain-of-thought, entered the vocabulary of anyone who writes prompts.

Why it matters to you

It is the explanation for something you probably already do. When an assistant hands you a wrong total, a hasty classification or a stat pulled from thin air, the simplest defence is to make it work in the open: list the steps, show the calculation, then conclude. This paper says why that gesture helps, and above all when it does nothing. Knowing the boundary stops you wasting words on small models, where the trick has no bite, and stops you trusting the steps you read too much: they look like a proof, but they are not always one.

What the paper says

The idea is minimal. In the prompt, instead of showing the model only examples of question and answer, you show it examples where between the question and the answer sits the reasoning written out in full: “Roger has 5 balls, he buys 2 packs of 3, so 5 plus 6 makes 11.” A few examples of this kind are enough (the paper uses eight) for the model to imitate the style and, faced with a new question, to write out the steps itself before answering.

The result that made noise is on GSM8K, a set of grade-school maths word problems, the multi-step kind where getting one piece wrong ruins everything. With just the reasoned examples added, the largest model of the time (PaLM, 540 billion parameters) reaches the outright top score on that benchmark, beating even a GPT-3 fine-tuned for the task and paired with an external verifier. On the largest models, on the hardest problem of the batch, the score more than doubles compared with normal prompting. The gain also shows up on arithmetic, commonsense and symbolic reasoning, and it grows the more steps the problem has: on single-step questions the difference is small, on multi-step ones it is large.

The most important point of the paper is not the number, it is the condition. Chain-of-thought is emergent with scale: below a certain model size it does not help, and sometimes makes things worse, because the small model writes a disjointed piece of reasoning and then leans on it. Above the threshold (in the paper, around a hundred billion parameters for the models of the day) the benefit appears and grows. It is not a recipe that always works: it is an ability that switches on once the model is big enough.

How much to trust it

The paper is solid on what it measures and honest about its edges. Three things are worth keeping in mind before generalising.

First: the numbers are from 2022, on models of that generation. Today’s models often reason in steps on their own, without you asking, and some have a dedicated reasoning mode. The marginal gain of “reason step by step” on a modern model can therefore be much smaller than the one reported here: the technique still holds as a principle, but do not expect the same jumps.

Second: written reasoning is no guarantee. The model produces a plausible chain of steps, and it can reach the right answer with wrong steps, or the wrong answer with steps that look correct. Reading the steps helps you check, but they are not a proof: they are text the model generated like everything else. Treat them as a trail to verify, not as evidence.

Third is a point of authorship that helps you not pick the wrong technique. This paper introduces chain-of-thought with examples: you show it a few pieces of reasoning done well, and it imitates them. The little phrase everyone knows, “think step by step” added to a question without examples, comes from a follow-up in the same line of work (Kojima and colleagues, also 2022): it works, but it is a slightly different thing and generally less powerful than reasoned examples. When a task really matters, showing two or three steps done well almost always beats a bare invitation to think it over.

What to do with it

The practical rule that comes out of the paper is short. When the task has several steps (a calculation, an estimate, a classification with criteria, a deduction from scattered facts) ask the model to write the steps before the conclusion, and if you can, show it an example of the reasoning you expect. You will get better answers and, just as useful, checkable ones: if the total is wrong, you see which step it went off at.

What not to conclude. That the model “reasons” like a person: it is continuing a text in a style you showed it, and the fact that the text resembles reasoning does not mean reasoning is happening inside. And do not conclude that more steps are always better: on a trivial question, forcing the model to unroll a chain of reasoning adds words, cost and chances to slip, without improving the answer. Chain-of-thought is a lever for hard problems, not a seasoning to put on everything.

Where to go deeper

The chapter “Instructions that work” in the pro track turns this result into a way of writing requests, and “Trusting the right amount” tackles the trap of steps that convince but do not prove.