ThinkTwice: Redefining AI Reasoning with a Two-Phase Approach

In the rapidly advancing field of artificial intelligence, ThinkTwice emerges as a standout framework. It employs a two-phase training process to tackle reasoning problems and refine solutions. Developed using Group Relative Policy Optimization (GRPO), this approach challenges the conventional methods in AI model training.

The Mechanics of ThinkTwice

At the core of ThinkTwice lies its dual-phase optimization strategy. Initially, the model focuses on solving reasoning problems. Following this, it refines its solutions to the same problems. This dual focus doesn't rely on external correctness signals or critique annotations. Instead, it leverages a binary correctness reward throughout both phases. Notably, this approach is applied across five mathematical reasoning benchmarks and two model families, including Qwen3-4B and Olmo3-7B.

The results speak volumes. In particular, Qwen3-4B shows impressive improvements. ThinkTwice outperforms GRPO on the AIME benchmark by 5 percentage points before refinement, soaring to an 11.5-point improvement after a self-refinement step, measured by pass@4. Western coverage has largely overlooked this, focusing instead on more mainstream technologies.

Training Dynamics and Impact

The training dynamics of ThinkTwice reveal an interesting pattern, an implicit rectify-then-fortify curriculum. Early in training, refinement primarily corrects errors. As the model improves, the focus shifts toward preserving correct solutions, leading to a refined reward signal. This methodology establishes joint training of reasoning and self-refinement as not just effective, but perhaps essential for strong AI development.

So, why does this matter? With AI systems increasingly tasked with complex problem-solving, an ability to refine and improve their own solutions is important. ThinkTwice offers a framework that not only enhances initial performance but elevates it through self-improvement.

Looking Forward

While ThinkTwice's results on mathematical benchmarks are impressive, the real question is, can this framework be extended to broader applications? If it can, it could redefine how we approach AI training across various domains. The benchmark results speak for themselves, but the potential applications are what truly excite.

In an industry that often chases the latest headline-grabbing innovation, ThinkTwice offers a methodical, yet transformative, approach to AI training. Western coverage has largely overlooked this, but the data shows that this framework might just be a major shift in AI development.