{"title":"What Are Reasoning Models?","description":"","content":"Ask GPT-4 a complex math problem and it answers immediately. Ask o1 the same question and something different happens. You see \"Thinking...\" for several seconds, sometimes longer. Then a detailed answer appears.\n\nWhat's happening during that thinking time?\n\no1 is a \"reasoning model.\" Instead of jumping straight to an answer, it generates an extended chain of reasoning internally. It considers approaches, checks its work, backtracks when stuck, and only then produces a final response.\n\nThis isn't just Chain of Thought prompting. Reasoning models are trained differently. They learn to reason through reinforcement learning, developing the ability to think through problems step by step without being explicitly told to.\n\nIn this article, I'll explain what reasoning models are, how they differ from regular LLMs, and when you should use them.\n\n---\n\n# The Difference: Prompting vs Training\n\nBefore reasoning models, we had Chain of Thought (CoT) prompting. Add \"Let's think step by step\" to your prompt, and regular LLMs produce reasoning traces.\n\nThis works, but it has limitations.\n\n#### Chain of Thought Prompting:\n\n- You must ask for reasoning explicitly\n- Quality depends on the prompt\n- Model wasn't trained to reason, just to follow instructions\n- Reasoning style varies based on how you ask\n\nReasoning models take a different approach:\n\n- Reasoning is built into the model through training\n- Extended thinking happens automatically\n- Model learned what good reasoning looks like\n- Consistent reasoning behavior without special prompts\n\nThe key difference is where the reasoning capability comes from. CoT prompting coaxes reasoning out of a model trained for general text generation. Reasoning models are specifically trained to reason well.\n\n### An Analogy\n\nThink of the difference between:\n\n1. **Asking someone to show their work** - They can do it if you ask, but it's not their natural mode. The quality varies.\n2. **Someone trained as a mathematician** - Showing work and checking steps is how they naturally think. It's internalized.\n\nReasoning models are like the mathematician. They've internalized the practice of systematic thinking through training.\n\n---\n\n# How Reasoning Models Work\n\nReasoning models generate extended \"thinking\" before producing an answer. This thinking can be visible or hidden.\n\n### Visible vs Hidden Thinking\n\n**OpenAI's o1**: The thinking process is mostly hidden. You see \"Thinking...\" but not the full reasoning chain. A summary may appear in the response.\n\n\n```shell\nUser: Solve this logic puzzle...\n\n[Thinking for 30 seconds - hidden from user]\n\no1: The answer is X. Here's why...\n```\n\n\n**DeepSeek R1**: The thinking process is visible. You can see the model's reasoning in `` tags.\n\n\n```shell\nUser: Solve this logic puzzle...\n\n\nLet me analyze this step by step.\nFirst, I notice that condition A implies...\nWait, that doesn't work because...\nLet me try a different approach...\nIf I consider B instead, then...\nYes, this leads to the solution.\n\n\nThe answer is X. Here's the reasoning...\n```\n\n\nBoth approaches have the same underlying mechanism: generate many reasoning tokens before committing to an answer. The difference is whether users see those tokens.\n\n### The Two Phases\n\nReasoning models operate in two distinct phases:\n\n\n```shell\n+------------------+ +-------------------+\n| Thinking Phase | --> | Response Phase |\n| (many tokens) | | (final answer) |\n+------------------+ +-------------------+\n\nThinking Phase:\n- Model generates reasoning tokens\n- Explores approaches, checks work\n- Can be hundreds or thousands of tokens\n- May or may not be visible to user\n\nResponse Phase:\n- Model produces final answer\n- Informed by all the thinking tokens\n- Usually more concise\n```\n\n\nThe thinking phase is where the magic happens. More thinking tokens generally means better answers on complex problems.\n\n---\n\n# What Makes a Problem \"Reasoning-Hard\"?\n\nNot every problem benefits from extended reasoning. Reasoning models excel on specific types of tasks.\n\n### Good for Reasoning Models\n\n#### **Complex math and logic:**\n\n\n```shell\n\"Prove that the square root of 2 is irrational.\"\n\nRegular LLM: May produce a flawed proof or skip steps\nReasoning model: Works through the proof systematically,\n checks each step, catches errors\n```\n\n\n#### **Multi-step puzzles:**\n\n\n```shell\n\"You have a 3-gallon jug and a 5-gallon jug.\nHow do you measure exactly 4 gallons?\"\n\nReasoning model explores possibilities:\n- Fill 5, pour into 3, leaves 2 in 5-gallon\n- Empty 3, pour 2 from 5 into 3\n- Fill 5, pour into 3 (which has 2), leaves 4 in 5-gallon\n```\n\n\n#### **Code with tricky edge cases:**\n\n\n```shell\n\"Write a function to detect cycles in a linked list.\"\n\nReasoning model thinks through:\n- What if list is empty?\n- What if cycle is at the head?\n- Floyd's algorithm: why does it work?\n- Let me trace through an example...\n```\n\n\n**Competition-level problems:**\n\nProblems from math olympiads, coding competitions, and logic puzzles where getting the right answer requires careful reasoning.\n\n### Not Ideal for Reasoning Models\n\n#### **Simple factual questions:**\n\n\n```shell\n\"What is the capital of France?\"\n\nReasoning model might overthink:\n\"Let me consider... France is in Europe...\nThe capital would be the seat of government...\nParis is the largest city...\"\n\nJust answer: \"Paris\"\n```\n\n\n#### **Creative writing:**\n\n\n```shell\n\"Write a poem about autumn.\"\n\nExtended reasoning can make output feel mechanical.\nCreative tasks benefit from intuition, not deliberation.\n```\n\n\n#### **Summarization and translation:**\n\nThese tasks don't have \"puzzle-like\" structure. There's no multi-step solution to discover.\n\n#### **High-throughput applications:**\n\nWhen you need to process thousands of requests per second, the latency and cost of reasoning models is prohibitive.\n\n---\n\n# How Reasoning Models Are Trained\n\nThe key innovation in reasoning models is how they're trained. It's not just fine-tuning on reasoning examples. It's reinforcement learning with verifiable rewards.\n\n### The Training Pipeline\n\nMost reasoning models follow a pipeline like this:\n\n\n```shell\n+----------------+ +---------+ +--------------------+\n| Pre-trained | --> | SFT | --> | Reinforcement |\n| Base Model | | | | Learning |\n+----------------+ +---------+ +--------------------+\n\nSFT = Supervised Fine-Tuning on reasoning examples\nRL = Reinforcement Learning with rewards for correct answers\n```\n\n\nThe reinforcement learning stage is critical. The model learns what reasoning strategies lead to correct answers.\n\n### Verifiable Rewards\n\nWhat makes RL work for reasoning? Verifiable problems.\n\nFor math problems, you can check if the answer is correct. For coding problems, you can run tests. This gives clear reward signals:\n\n\n```shell\nProblem: \"What is 17 x 23?\"\n\nModel attempt 1: \"Let me calculate... 17 x 20 = 340, 17 x 3 = 51,\n 340 + 51 = 391\"\nVerification: 17 x 23 = 391. Correct!\nReward: +1\n\nModel attempt 2: \"17 x 23 = 17 x 25 - 17 x 2 = 425 - 34 = 381\"\nVerification: 17 x 23 = 391, not 381. Wrong!\nReward: 0\n```\n\n\nThe model learns that certain reasoning patterns lead to correct answers. Over time, it internalizes effective reasoning strategies.\n\n### The DeepSeek R1 Discovery\n\nDeepSeek's R1 paper revealed something surprising: reasoning can emerge from pure RL without any supervised fine-tuning on reasoning examples.\n\nThey trained DeepSeek-R1-Zero starting from a base model, using only RL with accuracy rewards. No examples of reasoning traces. Just: \"here's a problem, here's whether your answer was right.\"\n\nThe model spontaneously developed reasoning behavior:\n\n\n```shell\nBefore RL training:\n\"What is 17 x 23?\" -> \"391\" (sometimes wrong, no work shown)\n\nAfter RL training:\n\"What is 17 x 23?\" ->\n\"\nI need to multiply 17 by 23.\nLet me break this down: 17 x 23 = 17 x (20 + 3)\n17 x 20 = 340\n17 x 3 = 51\n340 + 51 = 391\n\nThe answer is 391.\"\n```\n\n\nThe researchers called this the \"Aha moment.\" The model discovered on its own that showing intermediate steps helps it get correct answers. This is reasoning emerging from training, not from examples.\n\n---\n\n# The Reasoning Model Landscape\n\nSeveral reasoning models exist today, each with different characteristics.\n\n### OpenAI o1 and o3\n\n- Hidden thinking process\n- Very strong on math, coding, and science\n- More expensive than GPT-4\n- Likely uses RL + SFT + inference-time scaling\n\n\n```shell\no1-preview: First public reasoning model (Sept 2024)\no1: Improved version\no3: Latest, strongest version (announced Dec 2024)\n```\n\n\n### DeepSeek R1\n\n- Open-weight model (MIT license)\n- Visible thinking in tags\n- 671B parameters (mixture of experts)\n- Published detailed technical report\n\n\n```shell\nR1-Zero: Pure RL, no SFT (research demonstration)\nR1: Full model with SFT + RL\nR1-Distill: Smaller models trained on R1's outputs\n```\n\n\n### Claude's Extended Thinking\n\n- Available through extended thinking mode\n- Visible thinking process\n- Integrated into Claude 3.5 Sonnet and later\n\n### Comparison\n\n\n| ##### Model | ##### Thinking | ##### Open Weights | ##### Strengths |\n| --- | --- | --- | --- |\n| o1/o3 | Hidden | No | Math, coding, puzzles |\n| DeepSeek R1 | Visible | Yes | Math, coding, open research |\n| Claude Extended | Visible | No | General reasoning, coding |\n\n\n---\n\n# Inference-Time Scaling\n\nReasoning models benefit from inference-time scaling: spending more compute at inference to get better answers.\n\n### More Thinking = Better Answers\n\n\n```shell\nProblem complexity vs thinking time:\n\nSimple: \"2 + 2\" -> 1 second thinking -> correct\nMedium: \"Solve x^2 - 5x + 6 = 0\" -> 5 seconds -> correct\nHard: \"Prove Fermat's Last Theorem\" -> 60+ seconds -> better attempt\n```\n\n\nFor harder problems, more thinking tokens help. The model can:\n\n- Try multiple approaches\n- Catch and correct errors\n- Verify its answer\n\n### The Cost Trade-off\n\nMore thinking means more tokens, which means higher cost.\n\n#### Regular LLM:\n\n- Input: 100 tokens\n- Output: 50 tokens\n- Total: 150 tokens\n\n#### Reasoning Model:\n\n- Input: 100 tokens\n- Thinking: 500 tokens (may be billed at reduced rate)\n- Output: 50 tokens\n- Total: 650 tokens\n\nFor simple questions, this is wasteful. For hard problems, it's worth it.\n\n---\n\n## When to Use Reasoning Models\n\n### Use Reasoning Models For\n\n#### **Problems with verifiable answers:**\n\n- Math problems\n- Logic puzzles\n- Coding challenges\n- Scientific calculations\n\n#### **Tasks requiring multi-step reasoning:**\n\n- Planning and strategy\n- Debugging complex code\n- Analyzing edge cases\n- Proof construction\n\n#### **When accuracy matters more than speed:**\n\n- Important decisions\n- Production code review\n- Research analysis\n\n### Use Regular LLMs For\n\n#### **Simple queries:**\n\n- Factual questions\n- Definitions\n- Translations\n\n#### **Creative tasks:**\n\n- Writing\n- Brainstorming\n- Conversation\n\n#### **High-throughput applications:**\n\n- Chatbots\n- Content moderation\n- Quick classifications\n\n### A Simple Heuristic\n\nAsk yourself: \"Would a human need to think carefully and show their work to solve this?\"\n\n- Yes: Consider a reasoning model\n- No: A regular LLM is probably fine\n\n---\n\n# Limitations and Pitfalls\n\n### Overthinking\n\nReasoning models can overthink simple problems:\n\n\n```shell\nUser: \"Is 5 greater than 3?\"\n\nOverthinking model:\n\"Let me analyze this carefully.\n5 and 3 are both positive integers.\nOn the number line, 5 is to the right of 3.\nThe '>' symbol means 'greater than.'\nA number is greater than another if it's larger in value.\n5 - 3 = 2, which is positive.\nTherefore, 5 > 3.\nThe answer is yes.\"\n\nBetter: \"Yes\"\n```\n\n\nThis wastes tokens and can occasionally introduce errors through unnecessary complexity.\n\n### Cost and Latency\n\nReasoning models are significantly more expensive and slower:\n\n\n```shell\nGPT-4: ~$0.03 per 1K tokens, ~1-2 seconds\no1: ~$0.06 per 1K tokens, ~10-30 seconds\n\nFor 1000 queries:\nGPT-4: ~$30, ~30 minutes total\no1: ~$60+, ~8+ hours total\n```\n\n\n### Not Always Better\n\nOn some tasks, regular LLMs with good prompts outperform reasoning models:\n\n- Creative writing (reasoning makes it stilted)\n- Simple classification (overthinking hurts)\n- Tasks with ambiguous \"correct\" answers\n\n### Hallucination in Reasoning\n\nThe thinking process itself can contain errors. A model might reason confidently through wrong steps:\n\n\n```shell\n\nI recall that the capital of Australia is Sydney...\n[confident but wrong reasoning]\n\nThe capital of Australia is Sydney.\n```\n\n\nExtended thinking doesn't guarantee correct answers. It improves accuracy on average but isn't foolproof.\n\n---\n\n# Summary\n\nReasoning models represent a shift in how we build AI for complex problems.\n\n#### **What they are:**\n\n- LLMs trained to generate extended reasoning before answering\n- Built through reinforcement learning on verifiable problems\n- Generate more tokens internally to \"think through\" problems\n\n#### **How they differ from Chain of Thought:**\n\n- Reasoning is trained, not just prompted\n- Behavior is consistent without special instructions\n- Capability comes from RL, not just instruction-following\n\n#### **Key examples:**\n\n- OpenAI o1/o3 (hidden thinking)\n- DeepSeek R1 (open weights, visible thinking)\n- Claude Extended Thinking\n\n#### **When to use them:**\n\n- Complex math and logic\n- Multi-step puzzles\n- Coding with tricky edge cases\n- When accuracy matters more than speed\n\n#### **When to skip them:**\n\n- Simple factual questions\n- Creative tasks\n- High-throughput applications\n- Cost-sensitive use cases\n\nThe emergence of reasoning from pure RL training is one of the most significant findings in recent AI research. It suggests that with the right training signal, models can discover effective reasoning strategies on their own.\n\nFor complex problems where getting the right answer matters, reasoning models are a powerful tool. For everything else, regular LLMs with good prompting remain more practical.\n\n---\n\n# Further Reading\n\n- [DeepSeek-R1 Technical Report](undefined)\n- [Learning to Reason with LLMs (OpenAI)](undefined)\n- [Chain-of-Thought Prompting (Original Paper)](undefined)\n- [Scaling LLM Test-Time Compute](undefined)","pageType":"ai-engineering"}

What Are Reasoning Models?

Ashish Pratap Singh

Get Premium