针对网络舆情情感分析的RoBERTa-BiLSTM-Multihead Attention模型

RoBERTa-BiLSTM-Multihead Attention Model for Online Public Opinion Sentiment Analysis

摘要: 本文提出了一种基于RoBERTa-BiLSTM-Multihead Attention (RBMA)的融合模型，旨在解决当前网络舆情情感分析中存在的两个主要问题：一是单一模型难以充分提取文本的深层语义特征，二是传统方法在处理复杂情感转变和长序列依赖时表现不佳。为了克服这些挑战，本文首先使用RoBERTa预训练模型捕捉文本中词汇的语义信息，随后通过双向长短期记忆网络(BiLSTM)学习句子的正反向语义关系，以更好地理解上下文结构和文本的依赖关系。最后，模型引入多头注意力机制，对文本中的情感信息进行加权计算，以精准识别文本的情感倾向。通过在微博评论数据集上进行实验，RBMA模型展示了其在准确率、精确率、召回率和F1值等评价指标上的优异性能。相比于传统的LSTM、BERT、RoBERTa等模型，RBMA模型能够更全面地捕捉文本中的上下文信息，从而显著提升了情感分析的精度。进一步的实验证明，该模型在处理真实的舆情事件时，能够有效捕捉公众情感的波动，尤其在应对突发事件的舆情分析中表现出色。这为政府和企业的舆情管理提供了有力的技术支持。

Abstract: This paper proposes a hybrid model based on RoBERTa-BiLSTM-Multihead Attention (RBMA) to address two major challenges in online public opinion sentiment analysis: 1) the difficulty of a single model in fully capturing deep semantic features of text, and 2) the poor performance of traditional methods in handling complex sentiment shifts and long-sequence dependencies. To overcome these challenges, this study first employs the RoBERTa pre-trained model to capture semantic information at the lexical level. Then, a Bidirectional Long Short-Term Memory (BiLSTM) network is used to learn the bidirectional semantic relationships of sentences, thereby better understanding contextual structures and text dependencies. Finally, a Multihead Attention mechanism is introduced to assign weights to sentiment-related information in the text, allowing for more precise sentiment classification. Experiments conducted on a Weibo comment dataset demonstrate that the RBMA model achieves superior performance in accuracy, precision, recall, and F1-score compared to traditional models such as LSTM, BERT, and RoBERTa. The RBMA model more comprehensively captures contextual information within texts, significantly improving sentiment analysis accuracy. Further experiments confirm that the model effectively detects public sentiment fluctuations in real-world public opinion events, particularly excelling in the analysis of emergency or crisis-related online discussions. These findings provide strong technical support for public opinion management in both government and corporate settings.