Breaking the Jailbreak: Unmasking AI's Vulnerabilities in Finance

Jailbreaking isn't just a hacker's delight. It's a genuine hurdle for deploying Large Language Models (LLMs) and Vision Language Models (VLMs), especially in finance. The latter, with its dual processing of text and images, creates more attack points than a single-surfaced model. And yet, resources to detect these jailbreaks are as scarce as a good loot drop.

Enter FENCE

FENCE is here to change the game. It's a bilingual (Korean-English) dataset dedicated to training and evaluating jailbreak detectors specifically for finance. But it doesn't stop there. The dataset isn't just about the numbers. it emphasizes realism by pairing finance-related queries with image-grounded threats. Think of it as the ultimate training ground for AI's defensive line.

Why should you care? Simple. If your AI can't detect a jailbreak, it's like leaving your front door wide open. Vulnerabilities in these models could lay bare sensitive financial data, not just risking your economy but potentially shaking market confidence.

The Experiments and the Results

Experiments were conducted using both commercial and open-source VLMs. The results were eye-opening. While GPT-4o showed some resilience with measurable attack success rates, open-source models were left more exposed. Clearly, the game isn't over for AI developers.

But here's the kicker: a baseline detector trained on FENCE boasted a 99 percent in-distribution accuracy. It didn't just impress on home turf. it maintained strong performance on external benchmarks too. FENCE isn’t just another dataset. it's the cornerstone for training AI models to be more reliable in finance.

What Next for AI Safety?

FENCE is a step forward, but does the industry really understand its vulnerabilities? If AI can't defend itself, what's the point of deploying it in sensitive sectors? The game comes first. The economy comes second. Remember, another play-to-earn that forgot the play part won’t cut it here.

The path to safer AI in finance isn't just about creating more datasets. It's about fostering an environment where these models not only learn but adapt to threats in real-time. Retention curves don't lie. If an AI model can't retain its defenses, it won't last in the financial arena.