Breaking Down AI Red Teaming with Process Mining

In AI model evaluations, red teaming traditionally reduces adversarial campaigns to a binary metric: attack success rate (ASR). But this approach has always felt a tad reductive. Let's apply some rigor here. By ignoring the sequential dynamics of how models parry or succumb to attacks, significant nuances are lost. Enter process mining, a methodology borrowed from event logs analysis, promising a more granular view of these adversarial encounters.

A New Approach: Process Mining

The latest study attempts to fill this evaluative void by pitting 60 HarmBench prompts against two notable language models: GPT-OSS 120B and Llama 3.3 70B. With a battalion of 10 prompt mutation strategies, the models were each subjected to up to 110 attempts per prompt. The result? A staggering 8,575 scored events, dissected to extract Directly-Follows Graphs (DFGs) and state transition matrices. These tools unearth distinctive defense profiles that ASR alone would gloss over.

What they're not telling you: the findings are more revealing than a simple success rate could ever be. GPT-OSS appears to possess a near-absorbing refusal state, a stubborn barrier against adversarial attempts. In stark contrast, Llama's defenses seem riddled with escape routes, allowing jailbreaks with relative ease. The divergence in these defensive strategies is remarkable. It's a stark reminder that two models can exhibit radically different behaviors under pressure despite similar ASR outcomes.

Model-Specific Insights

What really stands out is the asymmetry in mutator effectiveness across the models. While one might assume a universal strategy for all, the truth is far messier. For instance, time-to-jailbreak distributions vary by an order of magnitude, underscoring that each model harbors its unique vulnerabilities and strengths. Color me skeptical, but the notion that a one-size-fits-all approach could suffice in adversarial testing is becoming increasingly untenable.

This kind of detailed analysis challenges the prevailing narrative in AI robustness. By focusing solely on ASR, we risk oversimplifying the complex interplay between models and their adversaries. Isn't it time we demanded more from our evaluations?

Why This Matters

The implications of this study stretch beyond academic curiosity. In a world where AI systems are becoming ever more integrated into decision-making processes, understanding how they might fail is key. As models are deployed in sensitive areas like security and autonomous systems, knowing their defense intricacies isn't just important. It's essential.

So, where does this leave us? The study highlights a critical need for diversified methodologies in evaluating AI defenses. The binary ASR metric simply doesn't survive scrutiny when faced with the complexities uncovered by process mining. Future evaluations must adopt more nuanced approaches to truly understand model defense mechanisms. After all, if our tools for assessment can't capture the true nature of model vulnerabilities, how can we claim to be advancing the field?