注意力机制引导的神经架构搜索在图像分类中的应用
Application of Attention-Guided Neural Architecture Search in Image Classification
DOI: 10.12677/csa.2024.1410197, PDF,    科研立项经费支持
作者: 许 斌, 刘健龙, 焦 旋：江西理工大学信息工程学院，江西 赣州；岑 颖：广东茂名幼儿师范专科学校，广东 茂名
关键词: 神经架构搜索；图像分类；瓶颈注意力模块；ShuffleUnit；Neural Architecture Search； Image Classification； Bottleneck Attention Module； ShuffleUnit
摘要: 神经架构搜索(Neural Architecture Search, NAS)是一种能够在预定义搜索空间内找到最优神经网络架构的方法，在图像分类等任务上具有重要的意义。注意力引导的架构搜索(Attention-guided Micro and Macro-Architecture Search, AGNAS)能根据注意力机制对重要操作赋予高的权重，避免了传统NAS方法中操作被错误选择的问题。本文分别从注意力机制和操作单元两个方面进一步对AGNAS方法进行改进。注意力机制方面，将通道注意力更换为瓶颈注意力(Bottleneck Attention Module, BAM)模块，即在通道注意力模块后增加空间注意力模块，以同时关注通道和空间重要特征。操作单元方面，引入ShuffleUnit操作，增加操作选择的多样性，进一步提升网络的特征表达能力。CIFAR-10数据集上的实验结果表明，与原始AGNAS相比，本文的改进方法能获得更低的分类错误率和更高的稳定性；与其他一些架构搜索方法相比，本文方法在分类错误率、参数量和搜索时间等方面的总体性能上更具优势。
Abstract: Neural Architecture Search (NAS) is a method that can find the optimal neural network architecture within a predefined search space and is of great significance in tasks such as image classification. Attention-Guided Micro and Macro-Architecture Search (AGNAS) can assign high weights to important operations based on attention mechanisms, avoiding the problem of incorrect selection of operations in traditional NAS methods. In this paper, we improve the AGNAS method from two aspects: attention mechanism and operation unit. Regarding attention mechanism, channel attention is replaced with bottleneck attention (BAM) module, which adds spatial attention module after channel attention module to focus on channel and spatial important features simultaneously. In terms of operation units, ShuffleUnit operation is introduced to increase the diversity of operation selection, further improving the feature expression ability of the network. The experimental results on the CIFAR-10 dataset show that compared with the original AGNAS, our method can achieve lower classification error and higher stability; Compared with the other architecture search methods, our method has overall advantages in classification error, parameters, and search time.
文章引用：许斌, 岑颖, 刘健龙, 焦旋. 注意力机制引导的神经架构搜索在图像分类中的应用[J]. 计算机科学与应用, 2024, 14(10): 1-9. https://doi.org/10.12677/csa.2024.1410197

参考文献

[1] Bello, I., Zoph, B., Vasudevan, V., et al. (2017) Neural Optimizer Search with Reinforcement Learning. Proceedings of Machine Learning Research, Sydney, 6-11 July 2017, 459-468.
[2] Real, E., Aggarwal, A., Huang, Y. and Le, Q.V. (2019) Regularized Evolution for Image Classifier Architecture Search. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 4780-4789. [Google Scholar] [CrossRef] 
[3] Pham, H., Guan, M., Zoph, B., et al. (2018) Efficient Neural Architecture Search Via Parameters Sharing. Proceedings of Machine Learning Research, Stockholm, 10-15 July 2018, 4095-4104.
[4] Liu, H., Simonyan, K. and Yang, Y. (2019) DARTS: Differentiable Architecture Search. ICLR International Confer-ence on Learning Representations, New Orleans, 6-9 May 2019, 1-13.
[5] Dong, X. and Yang, Y. (2019) Searching for a Robust Neural Architecture in Four GPU Hours. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 1761-1770. [Google Scholar] [CrossRef] 
[6] Li, G., Qian, G., Delgadillo, I.C., Muller, M., Thabet, A. and Ghanem, B. (2020) SGAS: Sequential Greedy Architecture Search. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 13-19 June 2020, 1617-1627. [Google Scholar] [CrossRef] 
[7] Xiao, H., Wang, Z., Zhu, Z., Zhou, J. and Lu, J. (2022) Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, 18-24 June 2022, 11882-11891. [Google Scholar] [CrossRef] 
[8] Xue, Y. and Qin, J. (2023) Partial Connection Based on Channel Attention for Differentiable Neural Architecture Search. IEEE Transactions on Industrial Informatics, 19, 6804-6813. [Google Scholar] [CrossRef] 
[9] Hu, X., Huang, L., Zeng, J., Wang, K. and Wang, Y. (2023) EGFA-NAS: A Neural Architecture Search Method Based on Explosion Gravitation Field Algorithm. Complex & Intelligent Systems, 10, 1667-1687. [Google Scholar] [CrossRef] 
[10] Wang, R., Cheng, M., Chen, X., et al. (2021) Rethinking Architecture Selection in Differentiable NAS. International Conference on Learning Representations, Vienna, 4-8 May 2021, 1-18. [Google Scholar] [CrossRef] 
[11] Sun, Z., Hu, Y., Lu, S., et al. (2022) AGNAS: Attention-Guided Micro and Macro-Architecture Search. Proceedings of Machine Learning Research, Baltimore, 17-23 July 2022, 20777-20789.
[12] Park, J., Woo, S., Lee, J., et al. (2018) BAM: Bottleneck Attention Module. British Machine Vision Conference, Newcastle, 17-18 July 2018, 1-14. [Google Scholar] [CrossRef] 
[13] Ma, N., Zhang, X., Zheng, H. and Sun, J. (2018) ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. Computer Vision—ECCV 2018, Munich, 8-14 September 2018, 122-138. [Google Scholar] [CrossRef] 
[14] Zoph, B., Vasudevan, V., Shlens, J. and Le, Q.V. (2018) Learning Transferable Architectures for Scalable Image Recognition. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 8697-8710. [Google Scholar] [CrossRef] 
[15] Liu, C., Zoph, B., Neumann, M., Shlens, J., Hua, W., Li, L., et al. (2018) Progressive Neural Architecture Search. Computer Vision—ECCV 2018, Munich, 8-14 September 2018, 19-35. [Google Scholar] [CrossRef] 
[16] Xu, Y., Xie, L., Zhang, X., et al. (2020) PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search. International Conference on Learning Representations, Addis Ababa, 26-30 April 2020, 1-13. [Google Scholar] [CrossRef]