基于多粒度注意力的可见光–红外行人重识别模态特异性算法
Multi-Granularity Modality Specific Attention Algorithm for Visible-Infrared Person Re-Identification
摘要: 可见光–红外行人重识别(Visible-infrared Person Re-identification, VI Re-ID)旨在根据裁剪后的查询行人样本，检索另一模态的画廊集中相同身份的行人样本。现有基于特征融合的方法主要采用双流特征提取模型来解决此任务，但由于两种模态的数据来源存在差异，不同模态特征提取分支存在特征粒度不一致的问题。为了解决这一问题，本文提出一种多粒度空间注意力模块，并进一步提出模态特异性双注意力模块，使用分散的细粒度注意力提取包含更多细节信息的可见光样本注意力描述符，使用集中的粗粒度注意力提取包含更多模糊信息的红外样本注意力描述符。实验证明融入多粒度方法的注意力模块有效的提高了双流特征提取模型在SYSU-MM01和RegDB数据集上的精度。
Abstract: Visible-infrared Person Re-identification aims to recognize pedestrian images with the same identity as query pedestrian image in the gallery set of another modality. Existing methods based on two-stream feature extraction network have been widely used to solve this task. However, due to the different data sources, the feature granularities in different modality feature extraction branches are inconsistent. To alleviate this issue, we propose a multi-granularity spatial attention model and further proposes a modality specific attention module, which use decentralized fine-grained attention to extract the attention descriptor of visible samples containing more detailed information, and use concentrated coarse-grained attention to extract the attention descriptor of infrared samples containing more fuzzy information. Extensive experiments show that our method greatly improves the performance of the end-to-end framework on two widely used visible-infrared Person Re-identification datasets, SYSU-MM01 and RegDB.
文章引用：王娅迪. 基于多粒度注意力的可见光–红外行人重识别模态特异性算法[J]. 计算机科学与应用, 2023, 13(1): 83-92. https://doi.org/10.12677/CSA.2023.131009

参考文献

[1] Sun, Y., Zheng, L., Yang, Y., Tian, Q. and Wang, S. (2018) Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline). Proceedings of the European Conference on Computer Vision (ECCV), Munich, 8-14 September 2018, 480-496. [Google Scholar] [CrossRef] 
[2] Zheng, L., Yang, Y. and Hauptmann, A.G. (2016) Person Re-Identification: Past, Present and Future.
[3] Zheng, Z., Zheng, L. and Yang, Y. (2017) Unlabeled Samples Generated by Gan Improve the Person Re-Identification Baseline in Vitro. Proceedings of the IEEE International Conference on Computer Vision, Venice, 22-29 October 2017, 3754-3762. [Google Scholar] [CrossRef] 
[4] Qian, X., Fu, Y., Jiang, Y.G., Xiang, T. and Xue, X. (2017) Mul-ti-Scale Deep Learning Architectures for Person Re-Identification. Proceedings of the IEEE International Conference on Computer Vision, Venice, 22-29 October 2017, 5399-5408. [Google Scholar] [CrossRef] 
[5] Moutafis, P., Leng, M. and Kakadiaris, I.A. (2016) An Overview and Empirical Comparison of Distance Metric Learning Methods. IEEE Transactions on Cybernetics, 47, 612-625. [Google Scholar] [CrossRef] 
[6] Deng, W., Zheng, L., Ye, Q., Kang, G., Yang, Y. and Jiao, J. (2018) Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-Identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-22 June 2018, 994-1003. [Google Scholar] [CrossRef] 
[7] Wu, A., Zheng, W. S., Yu, H. X., Gong, S. and Lai, J. (2017) RGB-Infrared Cross-Modality Person Re-Identification. Proceedings of the IEEE International Conference on Computer Vision, Venice, 22-29 October 2017, 5380-5389. [Google Scholar] [CrossRef] 
[8] Dai, P., Ji, R., Wang, H., Wu, Q. and Huang, Y. (2018) Cross-Modality Person Re-Identification with Generative Adversarial Training. IJCAI, 1, 6. [Google Scholar] [CrossRef] 
[9] Chen, Y., Wan, L., Li, Z., Jing, Q. and Sun, Z. (2021) Neural Feature Search for Rgb-Infrared Person Re-Identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, 20-25 June 2021, 587-597. [Google Scholar] [CrossRef] 
[10] Hao, Y., Wang, N., Gao, X., Li, J. and Wang, X. (2019) Dual-Alignment Feature Embedding for Cross-Modality Person Re-Identification. Proceedings of the 27th ACM Interna-tional Conference on Multimedia, Nice, 21-25 October 2019, 57-65. [Google Scholar] [CrossRef] 
[11] Liu, H., Tan, X. and Zhou, X. (2020) Parameter Sharing Explora-tion and Hetero-Center Triplet Loss for Visible-Thermal Person Re-Identification. IEEE Transactions on Multimedia, 23, 4414-4425. [Google Scholar] [CrossRef] 
[12] Zhang, X., Luo, H., Fan, X., Xiang, W., Sun, Y., Xiao, Q., Sun, J., et al. (2017) Alignedreid: Surpassing Human-Level Performance in Person Re-Identification.
[13] Ye, M., Shen, J., J Crandall, D., Shao, L. and Luo, J. (2020) Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification. In: European Conference on Computer Vision, Springer, Cham, 229-247. [Google Scholar] [CrossRef] 
[14] Liu, H., Cheng, J., Wang, W., Su, Y. and Bai, H. (2020) En-hancing the Discriminative Feature Learning for Visible-Thermal Cross-Modality Person Re-Identification. Neurocompu-ting, 398, 11-19. [Google Scholar] [CrossRef] 
[15] Wu, Q., Dai, P., Chen, J., Lin, C. W., Wu, Y., Huang, F., Ji, R., et al. (2021) Discover Cross-Modality Nuances for Visible-Infrared Person Re-Identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, 20-25 June 2021, 4330-4339. [Google Scholar] [CrossRef] 
[16] Huang, N., Liu, J., Luo, Y., Zhang, Q. and Han, J. (2023) Exploring Modality-Shared Appearance Features and Modality-Invariant Relation Features for Cross-Modality Person Re-Identification. Pattern Recognition, 135, Article ID: 109145. [Google Scholar] [CrossRef] 
[17] Fan, X., Luo, H., Zhang, C. and Jiang, W. (2020) Cross-Spectrum Dual-Subspace Pairing for RGB-Infrared Cross- Modality Person Re-Identification.
[18] Wang, G.A., Zhang, T., Cheng, J., Liu, S., Yang, Y. and Hou, Z. (2019) RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, 27 October-2 November 2019, 3623-3632. [Google Scholar] [CrossRef] 
[19] Wang, Z., Wang, Z., Zheng, Y., Chuang, Y.Y. and Satoh, S.I. (2019) Learning to Reduce Dual-Level Discrepancy for Infrared-Visible Person Re-Identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 618-626. [Google Scholar] [CrossRef] 
[20] Choi, S., Lee, S., Kim, Y., Kim, T. and Kim, C. (2020) Hi-CMD: Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 13-19 June 2020, 10257-10266. [Google Scholar] [CrossRef] 
[21] Wang, G. A., Zhang, T., Yang, Y., Cheng, J., Chang, J., Liang, X. and Hou, Z.G. (2020) Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 12144-12151. [Google Scholar] [CrossRef] 
[22] Woo, S., Park, J., Lee, J.Y. and Kweon, I.S. (2018) Cbam: Convolu-tional Block Attention Module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, 8-14 September 2018, 3-19. [Google Scholar] [CrossRef]