MedAgentAudit: Unpacking AI's Role in Medical Decision Making

Artificial intelligence is reshaping the medical landscape, but not always for the better. Large language models (LLMs) are being integrated into multi-agent systems that mimic multidisciplinary consultations. They promise to bring specialist roles, peer reviews, and consensus into clinical decision-making. Yet, as MedAgentAudit reveals, these systems often fall short.

The Core Problem

MedAgentAudit, a new audit framework, highlights a essential flaw in current AI evaluations. They focus on final accuracy rather than safety or transparency of the process. In a study of 3,600 execution logs, ten recurrent failure modes emerged. These span task comprehension, collaborative discussion, and decision-making.

This isn't just a technical issue. When 16.63% of cases in a study showed unsupported observations that propagated downstream, it underscores a larger problem. Can we trust AI if they repeat initial views in 98.42% of cases without re-examining evidence? The chart tells the story: consistency in error can be more dangerous than inconsistency.

Authority and Bias

MedAgentAudit also uncovers biases within AI systems. Authority bias was noted in 28.76% of the cases, jumping from 35.30% to 68.75% across rounds. This bias isn't just a statistic. It's a mirror reflecting how AI systems value perceived authority over hard evidence.

the failure to engage in specialist reasoning in 42.73% of cases raises another question. Are we building systems that prioritize speed over depth? Numbers in context: AI systems must evolve past superficial consensus.

Why It Matters

MedAgentAudit shifts the narrative from mere output accuracy to process-level safety and accountability. It's a call for transparency. In medicine, guessing isn't enough. Lives depend on accurate, evidence-based decisions.

With 14,400 cases analyzed across different architectures and datasets, the inconsistencies are stark. Collaboration yielded uneven accuracy gains and frequent process failures. These aren't just numbers, they're potential risks in a clinical setting.

So, should we trust AI in medicine? Yes, but cautiously. MedAgentAudit provides a practical foundation for transparent, auditable AI systems. It's a critical step toward clinician-supervised agentic systems where technology supports, rather than supplants, human expertise.