基于SAM的中药粉末显微图像分阶段框监督实例分割方法

A Staged Box-Supervised Instance Segmentation Method Based on SAM for TCM Powder Microscopy Images

摘要: 针对中药粉末显微图像中实例分割面临的目标密集分布、背景杂质干扰与矩形框标注缺乏像素级边界信息等问题，本文提出一种基于Segment Anything Model (SAM)的分阶段框监督实例分割框架。该框架依次包含三个阶段：首先，利用网格提示SAM自动生成高召回候选掩码，并通过面积约束去除明显碎片，解决目标密集场景下的实例召回问题；其次，构建形状与外观双分支候选拒识网络，外观分支以候选区域RGB图像与掩码为四通道输入，利用轻量卷积网络学习纹理判别特征；形状分支提取17维几何特征，包括紧致度、凸实度、Hu不变矩等，利用多层感知机学习轮廓判别特征；两分支在logit空间进行加权融合，有效抑制背景杂质和过分割碎片造成的假阳性；最后，利用内缩框构造伪掩码微调SAM的掩码解码器，以选择性补全不完整掩码，并引入精修前后掩码的自一致性交并比self-IoU门控机制防止精修漂移。在包含3191幅中药粉末显微图像、约44.7万个标注实例的数据集上的实验结果表明，本文方法在仅使用矩形框标注的条件下，F1分数达到0.863，实例级平均交并比mIoU达到0.809，显著优于多个主流方法。此外，消融实验也验证了各组件的独立贡献与有效性。

Abstract: To address the challenges of dense target distribution, background impurity interference, and the lack of pixel level boundary information from bounding box annotations in instance segmentation of traditional Chinese medicine powder microscopy images, this paper proposes a staged, box supervised instance segmentation framework built upon the Segment Anything Model. The framework comprises three stages. First, a grid prompted SAM automatically generates high recall candidate masks, and an area constraint removes obvious debris to address the instance recall problem in dense target scenarios. Second, a shape appearance dual branch candidate rejection network is constructed: the appearance branch takes a four channel input combining RGB image patches of candidate regions with their masks, and employs a lightweight convolutional network to learn texture discriminative features; the shape branch extracts a 17 dimensional geometric feature vector including compactness, solidity, Hu invariant moments, and so forth, and employs a multi layer perceptron to learn contour discriminative features. The two branches are fused in logit space via weighted summation, effectively suppressing false positives caused by background impurities and over segmentation fragments. Finally, shrunken box pseudo masks are leveraged to fine tune the SAM mask decoder for selective completion of incomplete masks, and a self IoU gating mechanism comparing masks before and after refinement is introduced to prevent refinement drift. Experimental results on a dataset containing 3191 TCM powder microscopy images with approximately 447,000 annotated instances demonstrate that, using only bounding box annotations, the proposed method achieves an F1 score of 0.863 and an instance level mean IoU of 0.809, significantly outperforming multiple mainstream methods. In addition, ablation studies validate the individual contributions and effectiveness of each component.