ABS-HDL：基于BIASRU的中文医学命名实体识别模型

ABS-HDL: Chinese Medical Named Entity Recognition Model Based on BIASRU

摘要: 中文医学命名实体识别旨在从中文非结构化医学文本中提取关键实体。针对模型训练时间长、传统字符向量处理方法容易忽视词边界等问题，提出了基于多头交互注意力的中文医学命名实体识别模型：ABS-HDL (ALBERT-BIASRU-SoftAttention-CRF Hybrid Deep Learning)。该方法首先使用ALBERT预训练模型分别获得词向量表示和字向量表示。其次，将字向量和词向量结合成一个字词向量矩阵。接着，本文提出了BIASRU语义提取层，通过将多头交互注意力融入到SRU中，实现了对字词向量矩阵特征的有效学习，并通过双向建模精确捕获序列上下文间的关系。此外，在软注意力机制权重分配层中，模型能够动态调整权重分配，增强了对实体边界的识别能力。最后，使用CRF解码层来优化标签序列的预测。实验结果表明，该模型在中文糖尿病数据集上与现有模型相比表现更好。

Abstract: Chinese medical named entity recognition aims to extract key entities from unstructured Chinese medical texts. Addressing issues such as the lengthy training time for models and the traditional character vector methods’ tendency to overlook word boundaries, a Chinese medical named entity recognition model based on multi-head interactive attention is proposed: ABS-HDL (ALBERT-BIASRU- SoftAttention-CRF Hybrid Deep Learning). This method initially employs the ALBERT pre-trained model to obtain separate word vector and character vector representations. Subsequently, it combines these vectors into a unified character-word vector matrix. Furthermore, this paper introduces the BIASRU semantic extraction layer, which integrates multi-head interactive attention into the SRU, effectively learning the features of the character-word vector matrix and precisely capturing the relationships within the sequence context through bidirectional modeling. Moreover, in the soft attention mechanism weight allocation layer, the model dynamically adjusts the distribution of weights, enhancing the ability to recognize entity boundaries. Lastly, a CRF decoding layer is used to optimize the prediction of the label sequence. Experimental results demonstrate that this model performs better on a Chinese diabetes dataset compared to existing models.