基于多头注意力机制的BiGRU-CNN文本情感分析
Text Sentiment Analysis Based on Multi-Head Attention Mechanism with BiGRU-CNN
摘要: 文本情感分析(Sentiment Analysis)当今已是自然语言处理(NLP)的热门研究领域和话题，该文提出一种基于多头注意力机制(Multi-head-Attention Mechanism)的双向门控循环网络(BiGRU)和卷积神经网络(CNN)的模型(Multi-head-Attention based on BiGRU-CNN model)。首先将文本用词向量表示，然后依次输入到BiGRU-CNN网络模型中得到本文的BiGRU全局特征和CNN局部特征；随后将BiGRU-CNN网络得到的特征输入到多头注意力机制层进行特征权重分配以获取文本有区分度且重要的特征信息；最后利用输出层中的Softmax函数对汽车评论文本进行情感极性分类。在汽车评论数据集中进行实验，准确率为90.76%，F1值为90.25%，实验结果与现有模型相比均有所提高。
Abstract: Text sentiment analysis is now a hot research field and topic in natural language processing (NLP). This paper proposes a new sentiment analysis model, which is based on the multi-head attention mechanism with Bidirectional Gated Recurrent Unit (BiGRU) and Convolutional Neural Network (CNN). First, the text review is represented by a word vector matrix, and then the BiGRU-CNN model is used to obtain the features input, and introduce the multi-head attention mechanism for feature weight distribution to obtain distinguished and important feature information of the text; finally classify the final sentiment polarity of the text by Softmax function. The experiment was performed on the car review data set, and the experiment achieved the accuracy rate of 90.76% and the F1 value of 90.25%. The experimental results were improved compared with the existing models.
文章引用：郝星跃. 基于多头注意力机制的BiGRU-CNN文本情感分析[J]. 计算机科学与应用, 2022, 12(1): 123-134. https://doi.org/10.12677/CSA.2022.121014

参考文献

[1] Nasukawa, T. and Yi, J. (2003) Sentiment Analysis: Capturing Favorability Using Natural Language Processing. Pro-ceedings of the 2nd International Conference on Knowledge Capture, Sanibel Island, 23-25 October 2003, 70-77. [Google Scholar] [CrossRef] 
[2] 李然, 林政, 林海伦, 等. 文本情绪分析综述[J]. 计算机研究与发展, 2018, 55(1): 30-52.
[3] 梁军, 柴玉梅, 原慧斌, 等. 基于深度学习的微博情感分析[J]. 中文信息学报, 2014, 28(5): 155-161.
[4] Irsoy, O. and Cardie, C. (2014) Opinion Mining with Deep Recurrent Neural Networks. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, 25-29 October 2014, 720-728. [Google Scholar] [CrossRef] 
[5] Kim, Y. (2014) Convolutional Neural Networks for Sentence Classifi-cation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, 25-29 Oc-tober 2014, 1746-1751. [Google Scholar] [CrossRef] 
[6] 刘龙飞, 杨亮, 张绍武, 等. 基于卷积神经网络的微博情感倾向性分析[J]. 中文信息报, 2015, 29(6): 159-165.
[7] Tang, D., Qin, B., Feng, X., et al. (2016) Effective LSTMs for Tar-get Dependent Sentiment Classification. Proceedings of Conference on COLING, Osaka, 11-16 December 2016, 3298-3307.
[8] Xiao, Z. and Liang, P.J. (2016) Chinese Sentiment Analysis Using Bidirectional LSTM with Word Embedding. Proceedings of the 2016 International Conference on Cloud Computing and Security, Nanjing, 29-31 July 2016, 601-610. [Google Scholar] [CrossRef] 
[9] Cho, K., Merrienboer, B.V., Gulcehre, C., et al. (2014) Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation.
https://arxiv.org/abs/1406.1078
[10] 缪亚林, 姬怡纯, 张顺, 等. CNN-BiGRU模型在中文短文本情感分析的应用[J]. 情报科学, 2021, 39(4): 85-91.
[11] Bahdanau, D., Cho, K. and Bengio, Y. (2014) Neural Machine Trans-lation by Jointly Learning to Align and Translate.
https://arxiv.org/abs/1409.0473
[12] 林原, 李家平, 许侃, 等. 基于多头注意力的双向LSTM情感分析模型研究[J]. 山西大学学报(自然科学版), 2020, 43(1): 1-7.
[13] 陈欣, 杨小兵, 姚雨虹, 等. 字词融合的双通道混合神经网络情感分析模型[J]. 小型微型计算机系统, 2021, 42(2): 279-284.
[14] Wei, P., Xu, N. and Mao, W. (2019) Modeling Conversation Structure and Temporal Dynamics for Jointly Predicting Rumor Stance and Veracity. Pro-ceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, 3-7 November 2019, 4789-4800. [Google Scholar] [CrossRef] 
[15] Pennington, J., Socher, R. and Manning, C.D. (2014) GloVe: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, 25-29 October 2014, 1532-1543. [Google Scholar] [CrossRef] 
[16] Devlin, J., Chang, M.W., Lee, K., et al. (2019) BERT: Pretraining of Deep Bidirectional Transformers for Language Understanding. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, Minneapolis, 2-7 June 2019, 4171-4186.
[17] Wang, J., Yu, L.C., Lai, K.R., et al. (2016) Dimensional Sentiment Analysis Using a Regional CNN-ISTM Model. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, 7-12 August 2016, 225-230. [Google Scholar] [CrossRef] 
[18] Chen, Q., Zhu, X., Ling, Z.H., et al. (2017) Enhanced LSTM for Nat-ural Language Inference. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, 30 July-4 August 2017, 1657-1668. [Google Scholar] [CrossRef] 
[19] Sun, X., Gao, Y., Sutcliffe, R., et al. (2019) Word Representation Learning Based on Bidirectional GRUs with Drop Loss for Sentiment Classification. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 99, 1-11.
[20] Zhang, H., Jin, W., Zhang, J., et al. (2017) YNU-HPCC at SemEval 2017 Task 4: Using a Multi-Channel CNN-LSTM Model for Sentiment Classification. Proceeding of the 11th Interna-tional Workshop on SEMANTIC Evaluation, Vancouver, 1 January 2017, 796-801. [Google Scholar] [CrossRef]