基于自相似结构特征和显著特征深度正交融合的图像检索

Image Retrieval with Deep Orthogonal Fusion of Self-Similarity Descriptor and Salient Features

摘要: 复杂场景下，由于图像内容复杂，细节信息丰富，以致深度学习网络提取的特征难以有效表达图像的重点信息。本文提出了融合正交显著特征和自相似描述符的图像检索模型。设计了自相似结构分支，获得图像局部自相似结构特征，将其编码为紧凑的自相似描述符，以有效描述图像内的结构信息；引入了注意力分支，将特征图中各通道相同位置的像素点作为一个向量，通过范数注意力生成包含显著特征的向量，通过自注意力和交叉注意力得到增强的显著特征。最后，引入了一个正交融合模块，融合结构特征和显著特征，从而得到复杂场景下图像的有效特征。实验证明，通过融合显著特征和结构特征，我们可以很好地提升基于全局表示的图像检索性能。

Abstract: In complex scenes, due to the intricate content and rich details of images, the features extracted by deep learning networks often fail to effectively represent the key information of the image. In this paper, we propose an image retrieval model that integrates orthogonal salient features and self-similarity descriptors. We design a self-similarity structural branch to obtain local self-similarity structural features of the image, which are encoded into compact self-similarity descriptors to effectively describe the structural information within the image. Additionally, an attention branch is introduced, where the pixels at the same position across all channels of the feature map are treated as a vector. Norm-based attention is used to generate a vector containing salient features, and enhanced salient features are obtained through both self-attention and cross-attention mechanisms. Finally, an orthogonal fusion module is introduced to combine the structural features and salient features, resulting in effective features for image retrieval in complex scenes. Experimental results demonstrate that by integrating salient features and structural features, we can significantly improve the performance of image retrieval based on global representations.