The Illusion of Security: Scrutinizing AI's Resistance to Prompt Attacks

In recent years, there's been a lot of noise around the vulnerability of computer-using agents (CUAs) to prompt-injection attacks. Researchers frequently tout success rates as high as 98%, but these figures are often cherry-picked from outdated models. What happens when we test these techniques against the latest models of today? Let's apply some rigor here.

Benchmarking the Present

Introducing CUA-HandCrafted, a new public benchmark that puts the spotlight on current frontier CUAs. Spanning 793 episodes across 24 web tasks, this benchmark evaluates 56 carefully designed attack templates. When tested against advanced models like Claude Sonnet 4.6 and GPT-5.4, these templates fell flat, achieving exactly zero multi-step attack successes out of 140 attempts. That's right, zero. Even when accounting for statistical variability, the upper bound for success hovers around a mere 2.60%.

However, the story doesn't end there. The same model weights crumble when faced with skill-injection attacks in a coding-agent benchmark known as SkillBench, where success rates rocket up to 100%. This stark contrast reveals a key point: the perceived security of these models doesn't generalize across different domains.

The Real Culprit

What they're not telling you: the high success rates frequently reported in literature are more about optimization than inherent vulnerability. The success of these attacks often relies on reinforcement learning-optimized text rather than the attack categories themselves. By withholding these optimized strings, researchers inadvertently make it impossible to reproduce their results. The claim doesn't survive scrutiny when extrapolated beyond the heavily targeted browser domain.

Beyond the Hype

Color me skeptical, but this pattern of focusing on outdated models and withholding key details does a disservice to the community. In reality, the security hardening of frontier models is highly domain-specific. They might hold up in certain conditions but falter dramatically in others. This discrepancy should serve as a wake-up call for those who believe in the invulnerability of AI systems.

So, here's the pointed question: Are we truly making progress in securing these agents, or are we just shuffling the vulnerabilities around, playing a game of cat and mouse? As we move forward, it's essential to not only diversify the testing domains but also share the precise methodologies openly. Otherwise, we'll be stuck in an endless loop of over-confidence followed by inevitable disappointment.