结合自注意力和归一化的MAC_BiLSTM文本分类模型

MAC_BiLSTM Text Classification Model Based on Self-Attention and Normalization

摘要: 针对文本分类任务中关键特征分布不均匀和双向长短期记忆网络(BiLSTM)局部特征信息提取不足的问题，提出了一种基于自注意力机制(Self_Attention)和归一化的多通道MAC_BiLSTM文本分类模型。在双向长短期记忆网络层之后加入自注意力机制并进行层归一化，同时将BiLSTM通道的信息与最初的词向量信息融合，输入卷积通道，再分别采用自注意力赋予词卷积方式重新计算后信息的词权重，并进行批归一化，重复两次之后再进行池化，最终将CNN通道池化后的特征信息与BiLSTM通道信息进行特征融合，并通过Softmax分类器得出分类结论。在模型的设计环境中，模型使用了更加平滑的Mish激活函数代替Relu，通过和其他深度学习模型在多个数据集上的比较，结果表明，所提出的模型与其他模型相比具有更好的分类效果。

Abstract: To address the problem of uneven distribution of key features in traditional text classification tasks and insufficient local feature extraction ability of Bi-directional Long Short-Term Memory (BiLSTM), a multi-channel text classification model based on self-attention mechanism and normalization is proposed. Self-attention mechanism and layer normalization are added after BiLSTM layer. The BiLSTM channel information and the initial word vector information are fused to input into the convolution channel. The weight of information after the convolution operation is given by self-attention, and the batch normalization is carried out. After repeated twice, the pooling is car-ried out. Then the feature information after max pooling and BiLSTM channel information are fused to input Softmax, and obtain the classification results. In the process of model operation, the pro-posed model uses smoother Mish activation function instead of Relu. Through comparison experi-ments with other deep learning models on multiple datasets, the results show that the proposed model has better classification accuracy than other models and has better classification perfor-mance.