Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Jason Wei, Xuezhi Wang, Dale Schuurmans, et al.2022NeurIPS 2022
Read it on arxiv.org(opens in a new tab)Why this one
Asking a model to write intermediate reasoning is easy to dismiss as prompt theater. The useful idea here is sharper: for large enough language models, extra generated tokens can act like scratch space, letting the model spend computation on a problem instead of jumping straight to an answer. People often copy the phrase "think step by step" and miss the engineering tradeoff. Chain of thought improves some tasks because it changes the shape of the computation, but it also costs latency, tokens, and can produce confident nonsense with a nicer transcript. Read this when you are deciding whether to solve a task with a bigger model, more retrieval, tool calls, or simply more room for the model to work.
What to take away
- Reasoning examples matter most when the task needs multiple intermediate steps, not simple recall.
- The gains appear mainly in large models, so the prompt pattern is not a free upgrade for every model size.
- Generated reasoning is useful compute, but it is also paid output that affects latency and serving cost.
Reads with
- Language Models are Few-Shot Learners
chain of thought is an in context learning trick that only appears after scale
- ReAct: Synergizing Reasoning and Acting in Language Models
turns the scratchpad into a loop of reasoning steps and external actions
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
shows the same extra test time compute idea trained into the model with reinforcement learning