ProofsNavigator:基于引导的可解释知识推理方法
ProofsNavigator: A Bootstrap Base Method for Explainable Knowledge Reasoning
DOI: 10.12677/csa.2024.147159, PDF,   
作者: 韦泽杨, 陈 涛*, 钟甫广:五邑大学电子信息工程学院,广东 江门;贾旭东:加州州立大学北岭分校计算机科学与工程学院,美国 洛杉矶
关键词: 知识推理证明生成逐步推理Knowledge Reasoning Proofs Generation Stepwise Reasoning
摘要: 让大规模语言模型生成推理步骤有助于构建可解释的知识推理系统。现有的知识推理方法可能生成不可靠并且与目标无关的推理步骤。为解决这一问题,该文提出一种基于引导的逐步推理方法ProofsNavigator。首先,使用通过Beam搜索生成多个候选推理步骤;然后,通过分别对候选推理步骤的有效性以及跟假设的相关性进行验证,挑选高质量的推理步骤;最后,将所选择的推理结论加入知识集中以进行下一轮循环。实验结果显示,该方法在三个难度依次递增的任务上的准确率分别为40.0%、35.6%和7.1%,比先前最优对比方法分别高1.1%、2.3%和0.2%。此外,该方法在标注数据较少的情况下仍能保持较好的性能。
Abstract: Generating reasoning steps with large-scale language models aids in constructing explainable knowledge reasoning systems. Existing methods for knowledge reasoning might produce unreliable and irrelevant reasoning steps. To address this issue, this article introduces a guided, step-by-step reasoning approach named ProofsNavigator. Initially, it generates multiple candidate reasoning steps through Beam search. Then, it selects high-quality reasoning steps by validating the validity of each candidate step and its relevance to the hypothesis. Finally, the selected reasoning conclusions are added to the knowledge set for the next iteration cycle. Experimental results show that this method achieves accuracies of 40.0%, 35.6%, and 7.1% on three tasks of increasing difficulty, respectively, outperforming the previous best methods by 1.1%, 2.3%, and 0.2%. Moreover, this method maintains good performance even with less annotated data.
文章引用:韦泽杨, 贾旭东, 陈涛, 钟甫广. ProofsNavigator:基于引导的可解释知识推理方法[J]. 计算机科学与应用, 2024, 14(7): 18-26. https://doi.org/10.12677/csa.2024.147159

参考文献

[1] Newell, A. and Simon, H. (1956) The Logic Theory Machine—A Complex Information Processing System. IEEE Transactions on Information Theory, 2, 61-79. [Google Scholar] [CrossRef
[2] Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., et al. (2018) HotpotQA: A Dataset for Diverse, Explainable Multi-Hop Question Answering. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, 31 October-4 November 2018, 2369-2380. [Google Scholar] [CrossRef
[3] Dua, D., Wang, Y., Dasigi, P., et al. (2019) DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning over Paragraphs. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, 2-7 June 2019, 2368-2378. [Google Scholar] [CrossRef
[4] Dalvi, B., Jansen, P., Tafjord, O., Xie, Z., Smith, H., Pipatanangkura, L., et al. (2021) Explaining Answers with Entailment Trees. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, 7-11 November 2021, 7358-7370. [Google Scholar] [CrossRef
[5] Saha, S., Ghosh, S., Srivastava, S. and Bansal, M. (2020) PRover: Proof Generation for Interpretable Reasoning over Rules. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, 16-20 November 2020, 122-136. [Google Scholar] [CrossRef
[6] Sun, C., Zhang, X., Chen, J., Gan, C., Wu, Y., Chen, J., et al. (2021) Probabilistic Graph Reasoning for Natural Proof Generation. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Online, 1-6 August 2021, 3140-3151. [Google Scholar] [CrossRef
[7] Liang, Z., Bethard, S. and Surdeanu, M. (2021) Explainable Multi-Hop Verbal Reasoning through Internal Monologue. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online, 6-11 June 2021, 1225-1250. [Google Scholar] [CrossRef
[8] Tafjord, O., Dalvi, B. and Clark, P. (2021) ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Online, 1-6 August 2021, 3621-3634. [Google Scholar] [CrossRef
[9] Yang, K., Deng, J. and Chen, D. (2022) Generating Natural Language Proofs with Verifier-Guided Search. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, 7-11 December 2022, 89-105. [Google Scholar] [CrossRef
[10] Qu, H., Cao, Y., Gao, J., Ding, L. and Xu, R. (2022) Interpretable Proof Generation via Iterative Backward Reasoning. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Seattle, 10-15 July 2022, 2968-2981. [Google Scholar] [CrossRef
[11] Ruis, L., Andreas, J., Baroni, M., et al. (2020) A Benchmark for Systematic Generalization in Grounded Language Understanding. Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, 6-12 December 2020, 19861-19872.
[12] Rae, J.W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., et al. (2021) Scaling Language Models: Methods, Analysis & Insights from Training Gopher. arXiv: 2112.11446. [Google Scholar] [CrossRef
[13] Chung, H.W., Hou, L., Longpre, S., et al. (2022) Scaling Instruction-Finetuned Language Models. arXiv: 2210.11416. [Google Scholar] [CrossRef
[14] Liu, Y., Ott, M., Goyal, N., et al. (2019) RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv: 1907.11692. [Google Scholar] [CrossRef