Calibrated Trust, Rigorous Validation, and the Real Barriers for the Use of Clinical Artificial Intelligence in Antimicrobial Stewardship

Review Article

Abstract

Antimicrobial resistance is a critical threat to global health, driven largely by suboptimal prescribing patterns. Although machine-learning algorithms have high predictive performance for infection risks and resistance profiles in silico, their clinical translation remains severely limited. This practical gap is frequently blamed on a deficit of clinician trust in black-box models, leading to a widespread demand for post-hoc explainable artificial intelligence. However, empirical evidence reveals that post-hoc explanations can mislead, inflating clinician confidence without verifying safety. The core challenge in clinical decision support is not trust itself but calibrated trust: ensuring clinical reliance matches the reliability of a system within a specific clinical environment. Post-hoc explainability methods often fail to reflect actual model mechanisms and can introduce unstable, non-causal associations. Unvalidated predictive systems, even when accompanied by plausible explanations, risk promoting automation bias and severe clinical errors. Clinical safety requires a fundamental shift in priority from post-hoc transparency to rigorous external validation, local calibration, and continuous post-deployment monitoring. Elevating validation and monitoring to primary clinical requirements, while treating explanation as an auxiliary, conditional tool, provides a mathematically and clinically sound model for artificial intelligence-enabled stewardship. Safe clinical adoption depends on establishing clear evidence of real-world utility rather than generating post-hoc rationalizations for unverified predictions