基于YOLOv26n双模型协同的舌象智能识别方法

An Intelligent Tongue Image Recognition Method Based on YOLOv26n Dual-Model Collaboration

摘要: 针对中医舌象图像识别中存在背景干扰明显、舌象类别视觉差异细微、多种舌象特征可能同时出现以及形态异常特征难以稳定识别等问题，提出一种基于YOLOv26n双模型协同的多标签舌象智能识别方法。首先利用YOLOv26n目标检测模型对输入图像中的舌体区域进行定位，获得舌体边界框并裁剪舌体感兴趣区域，以减少面部、嘴唇和背景区域对后续识别的影响；随后将舌体ROI输入改进后的YOLOv26n多标签分类模型，对正常舌、胖大舌、裂纹舌、薄白苔、黄厚苔、白厚苔、剥落苔、红点舌、淡白舌、淡红舌、黄厚腻苔、黑苔、地图舌和齿痕舌等舌象特征进行识别。考虑到一幅舌象图像可能同时包含舌色、舌苔和舌体形态等多种特征，本文将舌象分类任务建模为多标签识别问题，采用Sigmoid输出和加权二元交叉熵损失函数进行训练，并使用mAP、Micro-F1、Macro-F1和Hamming Loss等指标进行评价。针对齿痕舌等边缘形态异常特征，进一步引入几何形态学辅助分析方法，通过红色通道提取、阈值分割、轮廓提取、凸包分析和凸缺陷检测计算舌体边缘凹陷得分，并将几何得分作为辅助特征与深度学习模型输出进行融合。实验表明，该方法能够实现舌体检测、ROI提取、多标签舌象识别和边缘形态辅助分析，并可在树莓派5边缘端完成部署验证。所提出方法能够降低非舌体区域干扰，提高多种舌象特征共存场景下的识别能力，并为齿痕类形态异常提供可解释的辅助判断依据，可为中医舌象智能分析提供一种可行的深度学习实现方案。

Abstract: To address the problems of obvious background interference, subtle visual differences among tongue image categories, coexistence of multiple tongue features, and unstable recognition of morphological abnormalities in traditional Chinese medicine tongue image analysis, this paper proposes a multi-label intelligent tongue image recognition method based on YOLOv26n dual-model collaboration. First, a YOLOv26n object detection model is used to locate the tongue region in the input image. The tongue bounding box is obtained and the tongue region of interest is cropped, thereby reducing the influence of irrelevant regions such as the face, lips, and background on subsequent recognition. Then, the cropped tongue ROI is fed into an improved YOLOv26n multi-label classification model to identify tongue features, including normal tongue, enlarged tongue, fissured tongue, thin white coating, yellow thick coating, white thick coating, peeled coating, red spot tongue, pale tongue, light red tongue, yellow thick greasy coating, black coating, geographic tongue, and scalloped tongue. Considering that a single tongue image may simultaneously contain multiple features related to tongue color, tongue coating, and tongue morphology, the tongue classification task is formulated as a multi-label recognition problem. The model is trained using Sigmoid outputs and a weighted binary cross-entropy loss function, and evaluated using mAP, Micro-F1, Macro-F1, and Hamming Loss. For edge-related morphological abnormalities such as scalloped tongue, a geometric morphology-assisted analysis method is further introduced. This method calculates the tongue edge depression score through red channel extraction, threshold segmentation, contour extraction, convex hull analysis, and convexity defect detection. The geometric score is then fused with the output of the deep learning model as an auxiliary feature. Experimental results show that the proposed method can achieve tongue detection, ROI extraction, multi-label tongue feature recognition, and edge morphology-assisted analysis. It can also be deployed and verified on a Raspberry Pi 5 edge platform. The proposed method can reduce interference from non-tongue regions, improve the recognition ability in scenarios where multiple tongue features coexist, and provide interpretable auxiliary evidence for scalloped tongue-like morphological abnormalities. It offers a feasible deep learning-based solution for intelligent tongue image analysis in traditional Chinese medicine.