基于知识库标记预训练孪生神经网络的中文实体链接
Knowledge Marker-Based Pre-Trained Language Model with Siamese Network for Chinese Entity Linking
DOI: 10.12677/CSA.2022.124122, PDF,    科研立项经费支持
作者: 何展鹏：广东工业大学，计算机学院，广东 广州
关键词: 知识库问答；实体链接；预训练语言模型；Knowledge Base Question Answering； Entity Linking； Pre-Trained Language Model
摘要: 知识库问答(Knowledge Base Question Answering)有两个子任务：实体链接和关系预测。实体链接任务(Entity linking)是对于一个给定的文本，识别出其中实体指代(Mention)，并关联到知识库对应话题实体的过程。实体链接包括实体指代识别(MD)和实体消歧(ED)两个子任务。由于存在口语化严重、短文本信息少、实体多歧义多等问题，在中文数据集中更具挑战。传统方法并未充分利用知识库，缺少对短文本中指代和知识库实体的表示深入探究，本文提出用加入知识库标记的BERT模型进行实体识别，加入知识库标记及实体子图的BERT-SiameseFNN网络得到指代和候选实体的语义表示，进行实体消歧。通过在多个中文数据集上验证，表明该方法得到更充分利用知识库，并得到更好的匹配表示，有效提升实体链接性能。
Abstract: The knowledge base question answering (KBQA) task has two subtasks: entity linking and relation detection. Entity linking (EL) is the process of linking entity mentions appearing in given text with their corresponding topic entities in a knowledge base, which includes mention detection and entity disambiguation. Due to expression diversity, short context information, different meaning between similar candidate entities, it is more challenging in Chinese Entity Linking. Traditional methods do not make full use of the knowledge base and lack further exploration of representation between mention and candidate entity. We propose a knowledge marker method for both mention detection and entity disambiguation. In entity disambiguation, we also use a BERT-Siamese FNN network to encode mention-candidate entity pairs. The experimental results on two datasets show that the EKBERT reaches the state-of-the-art models and distills rich but discriminative information.
文章引用：何展鹏. 基于知识库标记预训练孪生神经网络的中文实体链接[J]. 计算机科学与应用, 2022, 12(4): 1202-1212. https://doi.org/10.12677/CSA.2022.124122

参考文献

[1] 韩先培, 等. CCKS2019知识库评测技术报告: 实体、关系、事件及问答[EB/OL]. 中文信息学报.
https://arxiv.org/pdf/2003.03875, 2017.
[2] Yu, M., Yin, W., Hasan, K.S., et al. (2017) Improved Neural Relation Detection for Knowledge Base Question Answering. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Volume 1, 571-581. [Google Scholar] [CrossRef] 
[3] Wu, P., Huang, S., Weng, R., et al. (2019) Learning Representation Mapping for Relation Detection in Knowledge Base Question Answering. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, July 2019, 6130-6139. [Google Scholar] [CrossRef] 
[4] Lai, Y., Feng, Y., Yu, X., et al. (2019) Lattice CNNS for Matching Based Chinese Question Answering. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 6634-6641. [Google Scholar] [CrossRef] 
[5] Duan, N. (2016) Overview of the NLPCC-ICCPOL 2016 Shared Task: Open Domain Chinese Question Answering. In: Natural Language Understanding and Intelligent Applica-tions, Springer, Cham, 942-948. [Google Scholar] [CrossRef] 
[6] Liu, A., Huang, Z., Lu, H., et al. (2019) BB-KBQA: BERT-Based Knowledge Base Question Answering. In: China National Conference on Chinese Computational Linguis-tics, Springer, Cham, 81-92. [Google Scholar] [CrossRef] 
[7] Devlin, J., Chang, M.W., Lee, K., et al. (2019) BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019, Min-neapolis, 2-7 June 2019, 4171-4186.
[8] 汪洲, 侯依宁, 等. 基于特征融合的中文知识库问答方法[EB/OL].
https://bj.bcebos.com/v1/conference/ccks2020/eval_paper/ccks2020_eval_paper_1_4_1.pdf, 2020.
[9] 2020全国知识图谱与语义计算大会评测及任务介绍新冠知识图谱构建问答[EB/OL]. http://sigkg.cn/ccks2020/?page_id=516
[10] Chakraborty, N., Lukovnikov, D., Maheshwari, G., et al. (2021) Introduc-tion to Neural Network-Based Question Answering over Knowledge Graphs. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 11, e1389. [Google Scholar] [CrossRef] 
[11] Wang, Y., Zhang, R., Xu, C. and Mao, Y. (2018) The APVA-Turbo Approach to Question Answering in Knowledge Base. Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, 20-26 August 2018, 1998-2009.
[12] Li, T., Liu, M., Zhang, Y., et al. (2021) A Review of Entity Linking Research Based on Deep Learning. Journal of Peking University: Health Sciences, 57, 91-98.
[13] Zhou, B., Sun, C., Lin, L., et al. (2018) LSTM Based Question Answering for Large Scale Knowledge Base. Journal of Peking University: Health Sciences, 54, 286-292.
[14] Yang, Y., He, X., Zhou, K., et al. (2019) Multi-Module System for Open Domain Chinese Question Answering over Knowledge Base.
[15] 张鸿志, 李如寐, 王思睿, 黄江华. 基于预训练语言模型的检索-匹配式知识库问答系统[EB/OL].
https://bj.bcebos.com/v1/conference/ccks2020/eval_paper/ccks2020_eval_paper_1_4_2.pdf, 2020.
[16] CCKS&百度2019中文短文本的实体链指第一名方案[EB/OL].
https://github.com/panchunguang/ccks_baidu_entity_link, 2019.
[17] Li, X.N., Yan, H., Qiu, X.P. and Huang, X.J. (2020) FLAT: Chinese NER Using Flat-Lattice Transformer. In: Proceedings of the 58th Annual Meeting of the Associa-tion for Computational Linguistics, Association for Computational Linguistics, Stroudsburg, 6836-6842.
https://aclanthology.org/2020.acl-main.611
[18] Reimers, N. and Gurevych, I. (2019) Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, 3-7 November 2019, 3982-3992. [Google Scholar] [CrossRef] 
[19] Liu, W.J., Zhou, P., Zhao, Z., et al. (2020) K-bert: Enabling Language Representation with Knowledge Graph. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 2901-2908. [Google Scholar] [CrossRef] 
[20] Xu, Y., Zhu, C., Xu, R., et al. (2020) Fusing Context into Knowledge Graph for Commonsense Reasoning.
[21] Wu, S. and He, Y. (2019) Enriching Pre-Trained Language Model with Entity Information for Relation Classification. Proceedings of the 28th ACM International Conference on In-formation and Knowledge Management, Beijing, 3-7 November 2019, 2361-2364. [Google Scholar] [CrossRef] 
[22] Conneau, A., Kiela, D., Schwenk, H., et al. (2017) Supervised Learning of Universal Sentence Representations from Natural Language Inference Data. Proceedings of the 2017 Con-ference on Empirical Methods in Natural Language Processing, Copenhagen, 7-11 September 2017, 670-680. [Google Scholar] [CrossRef] 
[23] Mueller, J. and Thyagarajan, A. (2016) Siamese Recurrent Architec-tures for Learning Sentence Similarity. 30th AAAI Conference on Artificial Intelligence, Phoenix, 12-17 February 2016, 2786-2792.
[24] Kim, Y. (2014) Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, October 2014, 1746-1751. [Google Scholar] [CrossRef] 
[25] CCKS2019&百度. CCKS2019中文短文本的实体链指[EB/OL].
https://www.biendata.xyz/competition/ccks_2019_el, 2019.