SOLE-R1: Redefining Robot Learning with Video-Language Models

Vision-language models (VLMs) have been making waves across various domains. Yet, their application in reinforcement learning (RL) often hits a snag with partial observability and distribution shift. The result? Models that fall short, leaving room for policies to exploit perceptual errors rather than genuinely solve tasks.

Introducing SOLE-R1

Enter SOLE-R1, a fresh take on tackling these limitations. Designed as a video-language reasoning model, it serves as the sole reward signal for online RL. The premise is simple yet revolutionary: provide only raw video observations and a natural-language goal, and let SOLE-R1 handle the rest.

Through spatiotemporal chain-of-thought (CoT) reasoning at every timestep, SOLE-R1 produces dense task progress estimates, directly usable as rewards. This approach is particularly compelling. It strips away the need for ground-truth rewards or task-specific tuning, offering a pure form of learning. Here's what the benchmarks actually show: SOLE-R1 enables zero-shot online RL from random initialization. Robots can now learn unseen manipulation tasks without prior demonstrations or success indicators.

A Look at the Training Pipeline

To train SOLE-R1, developers crafted a large-scale video trajectory and reasoning synthesis pipeline. This innovation generates temporally grounded CoT traces aligned with continuous progress supervision. Coupled with foundational spatial and temporal reasoning, the model is trained using a hybrid framework. This blends supervised fine-tuning with RL from verifiable rewards, ensuring the model's robustness and adaptability.

The result? SOLE-R1 excels in four distinct simulation environments and even a real-robot setting. It successfully tackles 24 unseen tasks, outshining some of the industry's strongest vision-language rewarders, including GPT-5 and Gemini-3-Pro. Notably, its resilience to reward hacking sets it apart, making it a standout in the field.

Why This Matters

So, why should we care? Because SOLE-R1 represents a significant leap forward for robot learning. It challenges the status quo by removing traditional limitations and opening up new possibilities for automation and AI. The architecture matters more than the parameter count here, and SOLE-R1 proves it.

Frankly, the industry can't overlook the potential impact of such a model. Are we on the brink of a new era where robots learn tasks more intuitively and efficiently than ever before? The numbers tell a different story, one where SOLE-R1 leads the way in reliable, adaptable learning.