Revolutionizing Code Editing: The Clean Pull Request Paradigm

Repository-level code editing has long been a challenge in machine learning. Models need to navigate complex dependencies and execute precise, multi-file edits across sprawling codebases. Recent progress, especially on benchmarks like SWE-bench, hinges on intricate agent architectures. But how much of this capability can we internalize within a model without such scaffolding?

Introducing Clean Pull Request

Enter the Clean Pull Request (Clean-PR) paradigm. It offers a fresh take on training, leveraging real-world GitHub pull requests as a novel signal for repository-level edits. The team behind Clean-PR has developed a scalable process to transform noisy pull request diffs into usable Search/Replace edit blocks. This process involves reconstruction and validation, culminating in a massive corpus of 2 million pull requests across 12 languages. It's the largest publicly available dataset of its kind.

Mid-Training with Real-World Data

Clean-PR's training pipeline involves a mid-training stage that simplifies the model's task. This is followed by a supervised fine-tuning process with error-driven data augmentation. The results? On the SWE-bench, the model shows spectacular performance. It outstrips the instruction-tuned baseline by 13.6% on the Lite version and 12.3% on the verified version.

This achievement demonstrates that repository-level understanding and editing can indeed be embedded into the model weights. And it does so under a streamlined, agentless approach. No heavy inference-time scaffolding needed. That's a big deal.

Implications for the Industry

If models can internalize such complex tasks, what's the future of agent-heavy architectures in code editing? The intersection is real. Ninety percent of the projects aren't. But those that are, like Clean-PR, could redefine our approach to AI-driven coding.

At the heart of this innovation is the question: Can simpler, more efficient methodologies replace the complex scaffolding that currently reigns in AI infrastructure? This shift could lead to more accessible, scalable, and cost-effective solutions in the industry.

Ultimately, Clean-PR isn't just a technical achievement. It's a bold statement about the direction AI development could take. Show me the inference costs. Then we'll talk.