解释性陷阱：目标检测模型中性能与解释可靠性研究

Explainability Traps: A Study on the Relationship between Object Detection Model Performance and Explanation Reliability

摘要: 目标检测模型在自动驾驶等安全关键领域广泛应用，其决策过程的可解释性对系统可靠性至关重要。当前研究重点主要集中在如何提高模型性能上，对于模型的可解释性以及解释质量与决策可靠性之间的内在关联关注较少。鉴于这一问题，本研究系统探究了目标检测模型性能与解释可靠性之间的关系。基于KITTI数据集，采用三种Faster R-CNN变体在简单、中等和困难三类场景下进行实验，通过Grad-CAM和SmoothGrad-IG两种解释方法，结合新提出的Energy-based Pointing Game和Performance-Explanation Correlation指标进行量化评估。结果表明：性能最佳的ResNet50_Epoch20在简单场景中PEC值为−0.189，揭示了“解释性陷阱”现象——高置信度预测反而伴随低质量解释；Grad-CAM生成的热力图分散且模型间差异显著，而SmoothGrad-IG产生的热力图高度收敛且模型间一致性高；简单场景中的解释可靠性问题最为严重，与直觉相反。

Abstract: Target detection model is widely used in key safety fields such as automatic driving, and the interpretability of its decision-making process is very important for system reliability. The current research focuses on how to improve the performance of the model, and pays less attention to the interpretability of the model and the internal relationship between the interpretation quality and decision reliability. In view of this problem, this study systematically explores the relationship between the performance of target detection model and interpretation reliability. Based on the Kitti data set, three fast R-CNN variants were used to carry out experiments in simple, medium and difficult scenarios. The two interpretation methods of grad cam and smooth grad Ig were combined with the newly proposed energy based pointing game and performance explanation correlation indicators for quantitative evaluation. The results show that the PEC value of resnet50_poch20 with the best performance is −0.189 in simple scenarios, which reveals the phenomenon of “interpretative trap”—high confidence prediction is accompanied by low quality interpretation; The thermal maps generated by grad cam are scattered and have significant differences among models, while the thermal maps generated by smooth grad IG are highly convergent and have high consistency among models; The problem of interpretation reliability is the most serious in simple scenarios, which is contrary to intuition.