AI in Drug Discovery: When Models Don't Cut It

AI systems are increasingly vital in scientific discovery. But what happens when the latest models don't measure up to the hype? Enter the Budget-Sensitive Discovery Score (BSDS), a new metric designed to evaluate AI selection strategies under budget constraints.

BSDS: The New Kid on the Block

BSDS isn't just any metric. It's a rigorously tested one with 20 theorems machine-checked by the Lean 4 proof assistant. This score penalizes false discoveries and excessive abstention based on budget, offering a strong way to assess AI's efficacy in scientific research.

Why should this matter to you? Because when you're spending big on experimental validation, you want to know your model's worth it. If nobody would play it without the model, the model won't save it.

LLMs Fail to Impress

In a recent study, 39 proposers were put to the test, including 11 mechanistic variants and 28 LLM configurations. The task? Select drug discovery candidates from MoleculeNet HIV and Tox21 datasets. Despite the buzz around LLMs, a simple RF-based Greedy-ML model outperformed all 28 LLM configurations.

No LLM, whether zero-shot or few-shot, could beat this baseline. In drug discovery where budgets and accuracy matter, LLMs added no marginal value. The game comes first. The economy comes second.

Beyond Hype: Real Results Matter

While everyone loves the idea of AI revolutionizing industries, reality often tells a different story. The BSDS framework shows that AI models must prove their worth beyond impressive headlines. Retention curves don't lie, and neither does the data.

So, what's the takeaway? AI isn't a silver bullet. In sectors like drug discovery, where precision and budget constraints are key, traditional methods still hold their ground. Before jumping on the latest AI trend, ask yourself: Is this truly better, or just the same old grind with a shiny new skin?