Keeping AI Smarter: The Battle Against Forgetfulness

If you've ever trained a model, you know the thrill of watching it master a task. But what if, along the way, it forgets what it's already learned? That's the catch with reinforcement learning with verifiable rewards (RLVR). While it's known for boosting large language model accuracy, it often overlooks a sneaky problem: previously solved tasks suddenly become unsolvable.

The Correct-Set Turnover Dilemma

Think of it this way: you've got a model that’s nailing new tasks, but it's quietly dropping the ball on old ones. This phenomenon is what researchers call 'correct-set turnover.' It's the dance of gaining new solutions while losing grip on the ones you had down pat.

Here's why this matters for everyone, not just researchers. As models like Qwen3-VL and Qwen2.5-Math go through their training cycles, they need to retain old solutions as much as they need to acquire new ones. Otherwise, they're just spinning their wheels without making real progress.

The Repair-Window Principle

The analogy I keep coming back to is fixing a leaky faucet. If you wait too long, the problem gets worse and the cost to fix it jumps. This is the core of the 'repair-window principle.' Models have a sweet spot timeframe where fixing forgotten prompts is cheap and easy. Miss it, and you pay the price in compute and time.

But the standard RLVR pipelines often miss this window. They’re too fixated on immediate gains to notice when things start slipping through the cracks.

Introducing a Retention-Aware Approach

Enter a new approach that’s got a bit of a spark. It's a retention-aware review mechanism designed to keep models from forgetting what they’ve learned. By periodically reintroducing mastered prompts, this method acts like a memory refresher, with zero additional rollout overhead. That’s right, no extra compute costs here.

Evaluations across 20 benchmarks, covering tasks from image-text to video, show this approach consistently outperforms GRPO, DAPO, and replay baselines. It’s like hitting the refresh button on your browser, ensuring everything is up to speed without lag.

So, What’s the Catch?

Honestly, the real question is why this wasn’t standard practice before. In a world where models are expected to handle an ever-growing list of tasks, why wouldn't we prioritize retention?

This isn’t just a technical tweak. It's a fundamental shift in how we think about model training and performance. Models that remember are more adaptable, more efficient, and ultimately more valuable.