Bridging the AI Theorem-Proving Gap with ECP

Mathematical competitions, those rigorous intellectual arenas, can usually be split into two categories: theorem proving and answer construction. Theorem proving calls for a proof of a given statement. Answer construction requires building an object that satisfies certain properties, along with proofs. Recent strides in large language models (LLMs) have made headway in theorem proving, yet answer construction lags behind. Enter the neurosymbolic approach called Enumerate-Conjecture-Prove (ECP).

The Two Sides of LLMs

General LLMs, for all their prowess in informal conjecturing, stumble generating reliable formal proofs. They're expensive and unreliable. On the other hand, prover LLMs, optimized for formal proofs, falter on the very front where mathematical reasoning is important: proposing candidate answers. Slapping a model on a GPU rental isn't a convergence thesis. This is where ECP steps in to fill the gap.

ECP leverages tool-assisted general LLMs to enumerate evidence and construct candidate answers. Meanwhile, it taps into prover LLMs to generate machine-checked proofs. It's an approach that's long overdue, one that doesn't just hack at the margins but aims to genuinely integrate capabilities where they're most needed.

Performance on the Benchmarks

On benchmarks like PutnamBench's and autoformalized MathArena's answer-construction problems, ECP flexes its muscle. It formally solves 17 out of 346 instances on PutnamBench and 18 out of 75 on MathArena, delivering admissible answers and proofs. That's a performance that overtakes LLM baselines with aligned inference budgets. Show me the inference costs. Then we'll talk.

But here's the question: why has formal answer construction remained underexplored until now? It exposes a blind spot in current AI research, where the push has been more about flashy conjecture than substantive proof integration. If the AI can hold a wallet, who writes the risk model?

Why It Matters

Why should this interest you? Because the intersection of AI and AI is real. Ninety percent of the projects aren't. ECP is part of that ten percent destined to change the way we handle formal problem-solving in mathematics. This isn't just about academic clout. As AI systems become more embedded in our decision-making, frameworks like ECP will be important for ensuring these decisions are grounded in verifiable logic, not just fancy number-crunching.

So, the next time you hear about advancements in AI theorem proving, remember ECP. It's not just another model, but a genuine attempt to bridge a significant gap in AI's mathematical capabilities.