Why Reinforcement Learning, Not Just Math, Is Powering the Next Leap in Language Models

For the past year, if you’ve been watching the progress of large language models (LLMs), you’ve probably noticed the math obsession. Leaderboards flooded with new records on MATH and AIME benchmarks. Chain-of-thought prompts everywhere. Math, it seems, became the gold standard for reasoning.

But while LLMs were getting better at solving equations, many weren’t getting better at… anything else.

A new paper from researchers at CMU ( Carnegie Mellon University ), UPenn, and others asked the exact question I’ve been wondering: Does tuning LLMs for math actually make them better in general?

Their answer is, Yes but only if you use Reinforcement Learning (RL), not Supervised Fine-Tuning (SFT). And the difference is bigger than you’d expect.

Not All Fine-Tuning Is Equal

To test how math-focused training affects broader skills, the researchers fine-tuned identical base models in two ways:

• SFT (Supervised Fine-Tuning): Feed the model expert solutions, make it imitate.
• RL (Reinforcement Learning): Let the model explore, then reward only correct final answers.