Can AI Prove the Unprovable? A Dive into FormalProofBench

In a world where AI models are increasingly tasked with solving complex problems, FormalProofBench has emerged as a key litmus test for gauging their capabilities in formal mathematics. Designed to challenge AI at the graduate level, this private benchmark focuses on producing formally verified mathematical proofs using Lean 4, a proof assistant language.

The Challenge of Formal Mathematics

FormalProofBench pairs natural-language problems with their corresponding formal statements in Lean 4. The task is straightforward in theory but daunting in practice: can AI models generate a Lean proof that passes muster with the Lean 4 checker? It's a question that gets to the heart of AI's current limitations and potential.

Covering topics like analysis, algebra, probability, and logic, the benchmark's problems are drawn from real-world academic scenarios. Think of it as a gauntlet for AI, drawn from qualifying exams and standard textbooks. Yet, the results are less than stellar. The best-performing models only manage a 33.5% accuracy rate. This isn't just a number, it's a statement on the current state of AI's mathematical inference abilities.

AI Models in the Trenches

We evaluated a range of latest models through an agentic harness, a method designed to maximize their problem-solving potential. Despite this, performance nosedives sharply after the top tier. This isn't a partnership announcement. It's a convergence of AI's reach and its current grasp.

Beyond accuracy, we also examined tool-use, failure modes, cost, and latency. The findings provide a comprehensive picture of where AI stands in the field of formal theorem proving. What does it reveal? That while AI can automate many tasks, formal proofs remain a steep hill to climb.

Why This Matters

So why should anyone outside of academia care about AI's struggles with formal proofs? The answer lies in the broader implications for AI applications. If these models can't handle formal proofs, what does that say about their readiness for tasks in fields like cryptography or automated reasoning, where precision is important?

The AI-AI Venn diagram is getting thicker, but success in one domain doesn't guarantee success in another. If agents have wallets, who holds the keys? The right proof at the right time could unlock new avenues of computational applications. Yet, as it stands, AI isn't quite there.

We're building the financial plumbing for machines, but for now, it seems that the pipes are still a bit leaky. The compute layer needs a payment rail, but perhaps we need to first ensure that the agents themselves can walk before they run.