Abstract
SWE-at-Scale synthesizes realistic software engineering tasks by inverting real-world fixes to generate diverse, executable bugs for evaluating coding agents.
As coding agents become increasingly capable, robust evaluation requires a broader and more diverse supply of verifiable software engineering (SWE) tasks. However, real-world tasks are constrained by the natural frequency of issue reporting and the substantial cost of manual curation. Furthermore, existing synthetic approaches rely primarily on the pretrained knowledge of large models coupled with heuristic prompting, often failing to capture the authentic, complex mechanics of software failures. To exponentially scale SWE task production, we introduce SWE-at-Scale, a framework that automatically synthesizes realistic issues by reverse-engineering real-world software fixes. Our key insight is that historical fixes encapsulate deep mechanistic knowledge of bug formation: inverting a fix transforms repair supervision into bug-induction supervision grounded in actual development scenarios. SWE-at-Scale transfers these learned bug patterns to novel code contexts, repeatedly sampling diverse bug typologies to construct executable SWE tasks. Extensive experiments show that SWE-at-Scale generates more diverse and realistic SWE tasks than current synthesis baselines while posing greater challenges to state-of-the-art coding agents. Notably, SWE-at-Scale can generate distinct bug types from a single code context, with bug diversity scaling robustly even up to 200 generations per file. Overall, SWE-at-Scale turns finite real-world fixes into reusable supervision for scalable synthesis of realistic, diverse, and challenging SWE tasks.
Community
What if we could turn bug fixes back into bugs? ๐
That's the idea behind SWE-at-Scale! We train a model to learn from real-world bug fixes and use it to create new bugs at scale. After all, if we want coding agents to get better at fixing bugs, why not teach another model to break things first?
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Improving Large Language Models for Code through Runtime Program-State Reasoning (2026)
- Schr"odinger's Code Repository: Have LLMs Learned SWE-bench or Memorized It? (2026)
- WideSWE: Can Coding Agents Coordinate Changes Across Repositories? (2026)
- SWE-Gate: Passing Functional Tests Is Not Enough for Software Engineering Agents (2026)
- SkillForge: Self-Distilling Agents for Project-Specific Issue Resolution (2026)
- RealSWE: A Compositional Evaluation of Coding Agents under Realistic User Requests (2026)
- SWEPatchTest: Evaluating Software Tests Beyond Reproduction (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2610.21359 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 1
Collections including this paper 0
No Collection including this paper