The explainable AI (XAI) research community faces a fundamental credibility crisis, according to a position paper released this week that argues the field has prioritized technique proliferation over theoretical rigor. While dozens of XAI methods—from LIME feature attributions to sparse autoencoders—have emerged in recent years, the paper contends these explanations are routinely generated and then discarded without meaningfully guiding actual workflows. The core problem: researchers develop explanations in isolation from the real-world contexts where they would be deployed, creating a disconnect between what XAI promises and what practitioners actually use. The authors argue that explainability research must shift from ad-hoc method development toward establishing robust foundational principles that guarantee explanations remain faithful to model behavior and genuinely serve human decision-makers.
Two complementary breakthroughs emerging alongside this critique offer pathways toward more grounded explainability research. IMEX (Interaction-Based Model Explanation) reframes explanation as an interactive process rather than a static output, recognizing that effective explanations must adapt to stakeholder questions and domain-specific reasoning patterns. Unlike traditional attribution methods that assign importance scores to individual features, IMEX models the dependencies and interactions between features, capturing how models actually make decisions through feature combinations—a fundamental insight that static explanations systematically miss. Simultaneously, HG-RAG (Hierarchy-Guided Retrieval-Augmented Generation) demonstrates how structured knowledge organization can improve explanation quality in language models. Rather than retrieving context from flat document stores, HG-RAG leverages hierarchical knowledge graphs to surface contextually relevant information at multiple levels of abstraction, enabling large language models to generate explanations that reflect domain structure rather than superficial keyword matching. Both approaches recognize that meaningful explanations require deeper engagement with domain knowledge and user needs.
The stakes of this methodological reckoning are particularly acute in high-stakes domains like medical imaging and genomics, where current XAI methods have demonstrably failed to bridge explanation-practice gaps. In clinical radiology, studies show radiologists report that current feature attribution techniques rarely match their actual diagnostic reasoning—resulting in explanations that receive institutional sign-off but don't influence treatment decisions. Similarly, RegNetAgents, a new multi-agent framework for cancer genomics, highlights how structured, query-driven explanation architectures designed specifically for domain workflows outperform generic XAI techniques. These applications underscore the position paper's central thesis: explainability research must stop treating explanations as post-hoc embellishments and instead embed them within foundational principles that guarantee fidelity to both model mechanics and human reasoning. Until XAI research prioritizes this grounding, explanations will remain technically sophisticated yet practically inert—a sobering reminder that advancing AI responsibly requires more than algorithmic innovation.