基于双重注意力机制的多标签司法文本分类
Multi-Label Classification Based on Dual Attention Mechanism for Judicial Documents
摘要: 多标签文本分类问题是自然语言处理领域中的一项重要子任务。考虑到传统的多标签文本分类问题往往没有对标签的信息进行充分利用，本文针对司法领域文本处理过程中遇到的多标签分类问题，提出了一种基于双重注意力机制的网络模型，对标签的固有信息进行充分挖掘，并从标签语义注意力机制以及标签结构注意力机制这两个角度为文本的特征向量进行权重的分配，捕获标签与文本之间的潜在关系。为验证模型的有效性，本文设计了对比实验，结果表明，本模型在宏平均F1值、微平均F1值、综合F1值上均有明显的性能提升。
Abstract: The multi-label text classification problem is an important subtask of natural language processing. Considering that the traditional multi-label text classification problems often do not make full use of the information of the labels, this paper proposes a model based on dual attention mechanism for the multi-label text classification problem in the judicial field. The inherent information of the text is fully mined, and the weights are assigned to the feature vectors of the text from the two aspects of the label semantic attention layer and the label structure attention layer to capture the potential relationship between the label and the text. In order to verify the validity of the model, a comparative experiment is designed in this paper. The results show that the model has obvious performance improvement in macro-F1, micro-F1, and union-F1.
文章引用：郭绮雯, 王勇, 王瑛. 基于双重注意力机制的多标签司法文本分类[J]. 计算机科学与应用, 2022, 12(2): 465-472. https://doi.org/10.12677/CSA.2022.122047

参考文献

[1] Boutell, M.R., Luo, J., Shen, X., et al. (2004) Learning Multi-Label Scene Classification. Pattern Recognition, 37, 1757-1771. [Google Scholar] [CrossRef] 
[2] Read, J., Pfahringer, B., Holmes, G., et al. (2009) Classifier Chains for Multi-label Classification. Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II. Berlin, 2009, 254-269. [Google Scholar] [CrossRef] 
[3] Tsoumakas, G. and Vlahavas, I. (2007) Random k-Labelsets: An Ensemble Method for Multilabel Classification. In: Kok, J.N., Koronacki, J., Mantaras, R.L., Matwin, S., Mladenič, D. and Skowron, A., Eds., European Conference on Machine Learning, Springer, Berlin, Heidelberg, 406-417.
[4] Elisseeff, A. and Weston, J. (2001) A Kernel Method for Multi-Labelled Classification. Advances in Neural Information Processing Systems 14 [Neural Information Processing Systems: Natural and Synthetic, NIPS, Vancouver, 3-8 December 2001, 681-687.
[5] Clare, A. and King, R.D. (2001) Knowledge Discovery in Multi-Label Phenotype Data. In: De Raedt, L. and Siebes, A. Eds., Principles of Data Mining and Knowledge Discovery, Springer, Berlin, Heidelberg, 42-53. [Google Scholar] [CrossRef] 
[6] Zhang, M.L. and Zhou, Z.H. (2007) ML-KNN: A Lazy Learning Approach to Multi-Label Learning. Pattern Recognition, 40, 2038-2048. [Google Scholar] [CrossRef] 
[7] Kurata, G., Bing, X. and Zhou, B. (2016) Improved Neural Network-based Multi-label Classification with Better Initialization Leveraging Label Co-occurrence. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego, June 2016, 521-526. [Google Scholar] [CrossRef] 
[8] Chen, G., Ye, D., Xing, Z., et al. (2017) Ensemble Application of Convolutional and Recurrent Neural Networks for Multi-Label Text Categorization. 2017 International Joint Conference on Neural Networks (IJCNN). Anchorage, 14-19 May 2017, 2377-2383. [Google Scholar] [CrossRef] 
[9] Yang, P., Sun, X., Li, W., et al. (2018) SGM: Sequence Gen-eration Model for Multi-Label Classification. Proceedings of the 27th International Conference on Computational Lin-guistics, Santa Fe, 20-26 August 2018, 3915-3926.
[10] You, R., Dai, S., Zhang, Z., et al. (2018) Attention XML: Ex-treme Multi-Label Text Classification with Multi-Label Attention Based Recurrent Neural Networks. Computing Re-search Repository, 18, 17-27.
[11] 刘心惠, 陈文实, 周爱, 等. 基于联合模型的多标签文本分类研究[J]. 计算机工程与应用, 2020, 56(14): 111-117.
[12] Devlin, J., Chang, M.W., Lee, K., et al. (2019) Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, 2-7.
[13] Wang, D., Cui, P. and Zhu, W. (2016) Structural Deep Network Embedding. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, August 2016, 1225-1234. [Google Scholar] [CrossRef] 
[14] Xiao, C., Zhong, H., Guo, Z., et al. (2018) CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction.