DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Daya Guo, Dejian Yang, et al.2025arXiv 2025
Read it on arxiv.org(opens in a new tab)Why this one
Read this after Chain-of-Thought Prompting Elicits Reasoning in Large Language Models and Training language models to follow instructions with human feedback. The important move here is not that a model writes longer thoughts, but that useful search behavior can be grown by rewarding final outcomes on tasks with checkable answers. People often reduce R1 to a product story, a benchmark jump, or a recipe for making models verbose. The deeper lesson is sharper: if you can build a reliable verifier, you may not need humans to write the path. That changes how you think about data, evaluation, and model improvement. It also explains why math and code were the first places this worked so well, and why open-ended domains are harder. Worth your evening because it turns reinforcement learning from an RL course artifact into a practical design pattern for making language models reason.
What to take away
- Outcome rewards can teach multi-step reasoning without copying human-written chains of thought.
- The method works best where answers are cheaply and reliably verifiable, like math and code.
- Distillation turns expensive emergent reasoning behavior into smaller, usable models.
Reads with
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
prompted reasoning is the behavior r1 tries to make trainable rather than coaxed
- Training language models to follow instructions with human feedback
the same alignment loop is repurposed from human preference to verifiable task rewards
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model
it is the contrasting path that removes reward model rl instead of leaning into outcome rewards