Unraveling the Origins: ATLAS Takes on RLVR Data Provenance

The rise of Reinforcement Learning from Verifiable Rewards (RLVR) datasets has introduced a complex challenge: provenance collapse. This occurs when the lineage of datasets isn't clear, causing a muddle that researchers and developers find increasingly difficult to navigate. But there's a new player in town that promises to clear the fog: Atomic-source Tracing via Lineage-Aware Search, or simply ATLAS.

Tracing the Origins

ATLAS systematically traces RLVR datasets back to their atomic sources. Remarkably, it attributes over 99.7% of 1.45 million instances to just 20 atomic sources. This kind of precision helps identify that most RLVR datasets aren't as unique as they seem. Many are permutations of a few shared upstream sources, and only a handful actually introduce new data.

This discovery is an eye-opener. It makes one wonder, why have we been accepting so much redundancy in our datasets? The problem isn't just redundancy, though. Data contamination risks lurk in these murky origins, threatening the integrity of our machine learning endeavors.

Enter DAPO++

In response to these findings, the creators of ATLAS didn't just stop at analysis. They went a step further, curating a new RLVR dataset called DAPO++. Crucially, DAPO++ is designed from a lineage-aware perspective, aiming to eliminate contamination and concentrate learning signals.

To achieve this, the team introduced Source-level Counterfactual Attribution (SCA). This principle measures the marginal utility of a sample by contrasting per-atomic-source RL checkpoints against a shared base model. It's a rigorous approach, ensuring that only truly valuable data makes it into the training set.

Quality Over Quantity

The results are impressive. The dataset quality score, Q, correlates strongly with the downstream performance of RLVR models. Testing on the Qwen3 series models has shown that DAPO++ consistently enhances performance on held-out benchmarks. This isn't just about having more data, it's about having the right data.

Why should this matter to those following AI developments? Western coverage has largely overlooked this nuance in dataset quality. The benchmark results speak for themselves. With a reliable predictor like Q, researchers can forego the trial and error of selecting training datasets and focus on more impactful work.

ATLAS, with its meticulous tracing and decontaminated dataset creation, sets a new standard. Isn't it time the rest of the AI community took notice?

For those interested in diving deeper, the code and data are available on GitHub. It's a chance to not only verify these claims but to contribute to a cleaner, more efficient future for RLVR datasets.