Bridging the Gap: Why Current XAI Fails Autonomous Driving Safety

Safety in autonomous driving is critical, but the evidence required to assure it's often mismatched with the tools we've. The documents show that while standards call for directed cause-and-effect chains, SHAP, the most widely recommended explainable AI (XAI) method, falls short, offering instead a ranked feature list that can't be directly translated into what safety standards demand.

The Evidence-Type Gap

This mismatch is termed the 'evidence-type gap.' It highlights a critical issue: the tools trusted to ensure safety don't produce the kind of evidence needed. From AMLAS to ISO certifications, 19 testable criteria across seven lifecycle stages are identified, yet only causal XAI methods seem up to the task in key phases like hazard identification and incident investigation.

Why should this concern us? Because the stakes are high. When your life is in the hands of an autonomous vehicle, you want more than a list of features. You want a guarantee that every decision the vehicle makes is backed by clear, causally linked evidence.

Causal Methods as the Solution

The study reveals causal XAI is structurally required to satisfy safety criteria, particularly in hazard identification (where it closes a 62% rubric gap) and incident investigation (50% gap). It's a call to action: shift focus from popular methods to those that meet the demands of safety standards.

Yet, despite their potential, even causal methods face challenges. They're necessary but not sufficient. The system was deployed without the safeguards the agency promised, and validating specificity remains an open challenge. Is the industry ready to prioritize safety over popularity?

The Path Forward

It’s time to rethink how we select XAI methods for safety assurance. Lifecycle-stage evidence demand should drive this choice, not method popularity. A proof of concept using nearly 2,000 real-world driving clips confirms that method selection can align with rubric predictions when guided by evidence needs.

Accountability requires transparency. Here's what they won't release: the details of how these methods are chosen and validated. Without transparency, the gap between what's needed and what's provided will remain. The affected communities weren't consulted in these decisions, and it’s their safety on the line.

autonomous driving, causality isn't just a theoretical pursuit. It's a safeguard. As this study makes clear, only by closing the evidence-type gap can we hope to achieve the safety standards we claim to uphold.