基于视觉–语言联合建模与LoRA微调的医疗废弃物检测模型

Medical Waste Detection Model Based on Vision-Language Joint Modeling and LoRA Fine-Tuning

摘要: 针对现有医疗废弃物分类模型在开放场景下存在小目标漏检率高、多类别混淆严重等问题，本文提出融合视觉–语言联合建模的改进型GroundingDINO模型。为了增强有效特征和精确位置信息的提取，并减少无效信息的干扰，在模型中构建了跨模态对比学习框架，结合低秩适配技术(Low-Rank Adaptation, LoRA)，对模型进行了轻量级优化，使其能够在保证高精度的同时，减少计算资源消耗。并引入EIoU (Enhanced IoU)损失函数，进一步提升了目标框的定位精度，并增强了模型在复杂医疗废弃物分类任务中的鲁棒性。结果表明，在依据国家医疗废弃物管理条例构建覆盖5大类20子类的医疗废弃物图像数据集上取得了良好效果，相比于基线模型GroundingDINO，以及阿里云发布的视觉理解大模型Qwen2.5-vl-72B，本实验基于GroundingDINO微调的GroundingDINO-MW在精确度、召回率、mAP以及F1指标上全面超越这几个检测模型。也充分证明了相较于原始模型可以更好地用在开放场景下的医疗废弃物分类识别中。

Abstract: To address the issues of high missed detection rate for small targets and severe multi-category confusion in existing medical waste classification models under open scenarios, this study proposes an improved GroundingDINO model incorporating visual-language joint modeling. The model constructs a cross-modal contrastive learning framework to enhance effective feature extraction and precise positional information while suppressing irrelevant interference. It implements lightweight optimization through Low-Rank Adaptation (LoRA) technology, achieving high accuracy with reduced computational resource consumption. The introduction of Enhanced IoU (EIoU) loss function further improves bounding box localization accuracy and enhances model robustness in complex medical waste classification tasks. Experimental results demonstrate superior performance in constructing a medical waste image dataset covering 5 major categories and 20 subclasses based on the national medical waste management regulations. Compared with baseline model GroundingDINO and Alibaba Cloud’s Qwen2.5-vl-72B visual understanding model, the fine-tuned GroundingDINO-MW based on GroundingDINO comprehensively outperforms these detection models in precision, recall, mAP, and F1 scores. This also fully validates that it can be better used in open-scenario medical waste classification and recognition compared to original models.