基于原型对比学习的可解释放射学报告生成方法

Interpretable Radiology Report Generation via Prototype-Guided Contrastive Learning

摘要: 自动放射学报告生成的目标是根据胸部X射线图像自动写出临床描述，以减轻放射科医生的报告撰写负担。已有方法虽取得一定进展，但生成结果缺乏可解释性，同时图像特征与文本语义之间没有显式对齐，限制了报告质量的提升。为此，我们提出ProtoR2Gen框架，引入一组可学习的疾病原型作为语义锚点，利用对比学习将图像特征向对应的原型聚集，从而在视觉编码和文本生成之间建立明确的对应关系。在推理时，原型的激活分布可转化为图像上的热力图，直观展示模型关注的解剖区域。在IU X-Ray数据集上，ProtoR2Gen的CIDEr指标达到0.656，显著优于对比方法。消融实验和可视化结果也证明了原型机制对语义对齐和可解释性的提升作用。

Abstract: Automatic radiology report generation aims to automatically produce clinical descriptions from chest X‑ray images to reduce the documentation burden on radiologists. Existing methods have made some progress, but the generated results lack interpretability, and visual features are not explicitly aligned with textual semantics, which limits further improvement in report quality. To address this, we propose ProtoR2Gen, a framework that introduces a set of learnable disease prototypes as semantic anchors. Through contrastive learning, image features are pulled toward their corresponding prototypes, establishing a clear correspondence between visual encoding and text generation. During inference, the activation distribution of prototypes can be converted into heatmaps over the image, intuitively showing which anatomical regions the model focuses on. On the IU X‑Ray dataset, ProtoR2Gen achieves a CIDEr score of 0.656, significantly outperforming comparison methods. Ablation studies and visualization results also demonstrate the effectiveness of the prototype mechanism in improving semantic alignment and interpretability.