Dissecting Prompt Injection: The Real Weakness in Frontier Models

In the race for solid large language models, prompt injection attacks are proving to be a formidable challenge. A recent analysis sheds light on the vulnerabilities of five frontier LLM agents, and the findings might surprise you. It's not about whether adversarial content gets seen. The real issue is how it's propagated across stages.

The Pipeline Problem

The study deployed cryptographic canary tokens across four kill-chain stages: Exposed, Persisted, Relayed, and Executed. This was done to evaluate where each model's defenses kicked in. Out of 764 total runs, 428 were attacked with no defenses. The result? Exposure was universal across all five models. The real variations emerged further downstream.

Claude vs. GPT-4o-mini: A Tale of Two Models

Among the models tested, Claude showed impressive resilience by stripping injections at the write_memory summarization stage, boasting a 0% attack success rate (ASR). On the other hand, GPT-4o-mini didn't fare as well, with a striking 53% ASR. The numbers tell a different story when you dig deeper.

Interestingly, DeepSeek showed a stark contrast in performance depending on the attack surface. It had a 0% ASR on memory surfaces but hit 100% on tool-stream surfaces. This complete reversal highlights the nuances of injection channels within the same model.

Active Defenses: A Mismatch

All four active defense conditions, write_filter, pi_detector, spotlighting, and their combination, fell short, resulting in a 100% ASR due to a mismatch with the threat-model surface. The architecture matters more than the parameter count here, and these numbers prove it.

So why should this matter to you? Because the very mechanisms meant to protect these models aren't as foolproof as advertised. If Claude's relay node can decontaminate downstream agents with 0 out of 40 canaries surviving into shared memory, then what role do active defenses really play?

What's Next?

Strip away the marketing and you get models that aren't as secure as they claim. The reality is, the industry needs better alignment between attack vectors and defense mechanisms. Does this mean we stop trusting our LLMs? Not quite. But it's a wake-up call for developers and researchers to focus on the stages where vulnerabilities truly lie. Frankly, it's time to rethink what model 'safety' really means.