基于语言知识的神经机器翻译研究进展
Advancements in Neural Machine Translation Research Based on Language Knowledge
摘要: 在本论文中,我们全面回顾了基于语言知识的神经机器翻译(Neural Machine Translation, NMT)方法,重点关注了如何将语言学特征和知识融入NMT系统中。首先,我们介绍了神经机器翻译的基本概念和发展背景,然后详细讨论了基于语言知识的神经机器翻译数据增强方面、翻译模型结构改进与外部语言知识的融合方面的各种方法。最后,我们总结了基于语言知识的神经机器翻译在实践中的优势与挑战,并展望了未来的研究方向。
Abstract: 在本论文中,我们全面回顾了基于语言知识的神经机器翻译(Neural Machine Translation, NMT)方法,重点关注了如何将语言学特征和知识融入NMT系统中。首先,我们介绍了神经机器翻译的基本概念和发展背景,然后详细讨论了基于语言知识的神经机器翻译数据增强方面、翻译模型结构改进与外部语言知识的融合方面的各种方法。最后,我们总结了基于语言知识的神经机器翻译在实践中的优势与挑战,并展望了未来的研究方向。
文章引用:张津一, 郭聪, 高忠辉. 基于语言知识的神经机器翻译研究进展[J]. 人工智能与机器人研究, 2023, 12(2): 97-106. https://doi.org/10.12677/AIRR.2023.122013

参考文献

[1] Bahdanau, D., Cho, K. and Bengio, Y. (2014) Neural Machine Translation by Jointly Learning to Align and Translate.
https://arxiv.org/abs/1409.0473
[2] Luong, M.T., Pham, H. and Manning, C.D. (2015) Effective Approaches to Attentionbased Neural Machine Translation. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, 17-21 September 2015, 1412-1421. [Google Scholar] [CrossRef
[3] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L. and Polosukhin, I. (2017) Attention Is All You Need. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, 4-9 December 2017, 5998-6008.
[4] 李洪政, 冯冲, 黄河燕. 稀缺资源语言神经网络机器翻译研究综述[J]. 自动化学报, 2021, 47(6): 1217-1231.
[5] Marlies, W., Bisazza, A. and Monz, C. (2017) Dynamic Data Selection for Neural Machine Translation. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), Copenhagen, 7-11 September 2017, 1400-1410.
[6] Zhang, P., Xu, X. and Xiong, D. (2018) Active Learning for Neural Machine Translation. 2018 International Conference on Asian Language Processing, Bandung, 15-17 November 2018, 153-158. [Google Scholar] [CrossRef
[7] Wang, R., Utiyama, M., Finch, A.M., Liu, L., Chen, K. and Sumita, E. (2018) Sentence Selection and Weighting for Neural Machine Translation Domain Adaptation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 26, 1727-1741. [Google Scholar] [CrossRef
[8] 宜年, 艾山∙吾买尔, 买合木提∙买买提, 等. 基于多种数据筛选的维汉神经机器翻译[J]. 厦门大学学报(自然科学版), 2022, 61(4): 660-666.
[9] 刘欢, 刘俊鹏, 黄锴宇, 等. 面向低资源俄汉机器翻译的领域适应方法[J]. 厦门大学学报(自然科学版), 2022, 61(4): 654-659.
[10] Sennrich, R., Haddow, B. and Birch, A. (2016) Improving Neural Machine Translation Models with Monolingual Data. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, 7-12 August 2016, 86-96. [Google Scholar] [CrossRef
[11] Caswell, I., Chelba, C. and Grangier, D. (2019) Tagged Back-Translation. Proceedings of the 4th Conference on Machine Translation, Florence, 1-2 August 2019, 53-63. [Google Scholar] [CrossRef
[12] Khatri, J. and Bhattacharyya, P. (2020) Filtering Back-Translated Data in Unsupervised Neural Machine Translation. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, 8-13 December 2020, 4334-4339. [Google Scholar] [CrossRef
[13] Wei, H., Zhang, Z., Chen, B. and Luo, W. (2020) Iterative Domain-Repaired Back-Translation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Online, 16-20 November 2020, 5884-5893. [Google Scholar] [CrossRef
[14] Abdulmumin, I., Galadanci, B.S. and Sinan, I.I. (2020) Iterative Self-Learning for Enhanced Back-Translation in Low Resource Neural Machine Translation. Proceedings of the 7th Workshop on Asian Translation, Suzhou, December 2020, 170-178.
[15] Pham, H., Wang, X., Yang, Y. and Neubig, G. (2021) Meta Back-Translation. arXiv: 2102.07847.
[16] Jiao, R., Yang, Z., Sun, M. and Liu, Y. (2021) Alternated Training with Synthetic and Authentic Data for Neural Machine Translation. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Online, 1-6 August 2021, 1828-1834. [Google Scholar] [CrossRef
[17] Zhang, J. and Matsumoto, T. (2019) Corpus Augmentation for Neural Machine Translation with Chinese-Japanese Parallel Corpora. Applied Sciences, 9, Article 2036. [Google Scholar] [CrossRef
[18] 尤丛丛, 高盛祥, 余正涛, 等. 基于同义词数据增强的汉越神经机器翻译方法[J]. 计算机工程与科学, 2021, 43(8): 1497-1502.
[19] 贾承勋, 赖华, 余正涛, 等. 基于短语替换的汉越伪平行句对生成[J]. 中文信息学报, 2021, 35(8): 47-55.
[20] Song, K., Zhang, Y., Yu, H., Luo, W., Wang, K. and Zhang, M. (2019) Code-Switching for Enhancing NMT with Pre-Specified Translation. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, 2-7 June 2019, 449-459.
[21] 朱俊国, 杨福岸, 余正涛, 等. 低频词表示增强的低资源神经机器翻译[J]. 中文信息学报, 2022, 36(6): 44-51.
[22] 张一鸣, 刘俊鹏, 宋鼎新, 等. 融合数据增强与多样化解码的神经机器翻译[J]. 厦门大学学报(自然科学版), 2021, 60(4): 670-674.
[23] Zhao, S., Wang, H., Liu, T. and Li, S. (2009) Extracting Paraphrase Patterns from Bilingual Parallel Corpora. Natural Language Engineering, 15, 503-526. [Google Scholar] [CrossRef
[24] Baziotis, C., Haddow, B. and Birch, A. (2020) Language Model Prior for Low-Resource Neural Machine Translation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Online, 16-20 November 2020, 7622-7634. [Google Scholar] [CrossRef
[25] 朱志国, 郭军军, 余正涛. 一种Mask交互融合预训练知识的低资源神经机器翻译方法[J/OL]. 小型微型计算机系统: 1-8.
http://kns.cnki.net/kcms/detail/21.1106.tp.20221123.1125.022.html
[26] 王可超, 郭军军, 张亚飞, 高盛祥, 余正涛. 基于回译和比例抽取孪生网络筛选的汉越平行语料扩充方法[J]. 计算机工程与科学, 2022, 44(10): 1861-1868.
[27] Ji, Y., Shi, L., Su, Y.L., Ren, Q., Wu, N. and Wang, H. (2021) A Strategy for Referential Problem in Low-Resource Neural Machine Translation. In: Farkaš, I., Masulli, P., Otte, S. and Wermter, S., Eds., Artificial Neural Networks and Machine Learning—ICANN 2021, Springer, Cham, 321-332. [Google Scholar] [CrossRef
[28] Song, K., Zhang, Y., Zhang, M. and Luo, W. (2018) Improved English to Russian Translation by Neural Suffix Prediction. Thirty-Second AAAI Conference on Artificial Intelligence, Riverside, 2-7 February 2018, 410-417.
[29] Xiao, Y.L., Liu, L.M., Huang, G.P., Qu, C., Huang, S.J., Shi, S.M. and Chen, J.J. (2022) BiTIIMT: A Bilingual Text-Infilling Method for Interactive Machine Translation. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Dublin, 22-27 May 2022, 1958-1969. [Google Scholar] [CrossRef
[30] Zhang, T., Ye, W., Yang, B.S., Zhang, L., Ren, X.Z., Liu, D., Sun, J., Zhang, S.K., Zhang, H.B. and Zhao, W. (2022) Frequency-Aware Contrastive Learning for Neural Machine Translation. Proceedings of the AAAI Conference on Artificial Intelligence, 36, 11712-11720. [Google Scholar] [CrossRef
[31] Currey, A. and Heafield, K. (2019) Incorporating Source Syntax into Transformer-Based Neural Machine Translation. Proceedings of the 4th Conference on Machine Translation, Florence, 1-2 August 2019, 24-33. [Google Scholar] [CrossRef
[32] Li, Y., Yin, Y., Li, J. and Zhang, Y. (2022) Prompt-Driven Neural Machine Translation. Findings of the Association for Computational Linguistics: ACL 2022, Dublin, 22-27 May 2022, 2579-2590. [Google Scholar] [CrossRef
[33] Sennrich, R. and Haddow, B. (2016) Linguistic Input Features Improve Neural Machine Translation. Proceedings of the First Conference on Machine Translation, Berlin, 11-12 August 2016, 83-91. [Google Scholar] [CrossRef
[34] Chen, K., Wang, R. and Utiyama, M. (2017) Neural Machine Translation with Source Dependency Representation. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, 7-11 September 2017, 2836-3842. [Google Scholar] [CrossRef
[35] Qing-dao-er-ji, R., Su, Y.L. and Liu, W.W. (2020) Research on the LSTM Mongolian and Chinese Machine Translation Based on Morpheme Encoding. Neural Computing and Applications, 32, 41-49. [Google Scholar] [CrossRef
[36] Zhang, J. and Matsumoto, T. (2017) Improving Character Level Japanese-Chinese Neural Machine Translation with Radicals as an Additional Input Feature. Proceedings of the 2017 International Conference on Asian Language Processing, Singapore, 5-7 December 2017, 172-175. [Google Scholar] [CrossRef
[37] Eriguchi, A., Hashimoto, K. and Tsuruoka, Y. (2016) Tree-to-Sequence Attentional Neural Machine Translation. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, 7-12 August 2016, 823-833. [Google Scholar] [CrossRef
[38] Chen, H., Huang, S., Chiang, D. and Chen, J. (2017) Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, 30 July-4 August 2017, 1936-1945. [Google Scholar] [CrossRef
[39] Gū, J., Shavarani, H. and Sarkar, A. (2018) Top-Down Tree Structured Decoding with Syntactic Connections for Neural Machine Translation and Parsing. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, 31 October-4 November 2018, 401-413. [Google Scholar] [CrossRef
[40] Pham, T., Macháček, D. and Bojar, O. (2019) Promoting the Knowledge of Source Syntax in Transformer NMT Is Not Needed. Computación y Sistemas, 3, 923-934. [Google Scholar] [CrossRef
[41] Chang, T. and Rafferty, A.N. (2020) Encodings of Source Syntax: Similarities in NMT Representations across Target Languages. Proceedings of the 5th Workshop on Representation Learning for NLP, Online, 9 July 2020, 7-16. [Google Scholar] [CrossRef
[42] Zhang, M., Li, Z., Fu, G. and Zhang, M. (2019) Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, 2-7 June 2019, 1151-1161. [Google Scholar] [CrossRef
[43] Wang, Y., Lee, H. and Chen, Y. (2019) Tree Transformer: Integrating Tree Structures into Self-Attention. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, 3-7 November 2019, 1061-1070. [Google Scholar] [CrossRef
[44] Moussallem, D., Arcan, M., Ngomo, A.N. and Buitelaar, P. (2019) Augmenting Neural Machine Translation with Knowledge Graphs.
https://arxiv.org/abs/1902.08816
[45] 潘一荣, 李晓, 杨雅婷, 等. 面向维汉机器翻译的层次化多特征融合模型[J]. 厦门大学学报(自然科学版), 2020, 59(2): 206-212.
[46] Duan, X., Ji, B., Jia, H., Tan, M., Zhang, M., Chen, B., Luo, W. and Zhang, Y. (2020) Bilingual Dictionary Based Neural Machine Translation without Using Parallel Sentences. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, 5-10 July 2020, 1570-1579. [Google Scholar] [CrossRef
[47] 满志博, 毛存礼, 余正涛, 李训宇, 高盛祥, 朱俊国. 基于多语言联合训练的汉-英-缅神经机器翻译方法[J]. 清华大学学报(自然科学版), 2021, 61(9): 927-935.
[48] Chen, K., Wang, R., Utiyama, M. and Sumita, E. (2022) Integrating Prior Translation Knowledge into Neural Machine Translation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30, 330-339. [Google Scholar] [CrossRef
[49] Lu, Z., Li, X., Liu, Y., Zhou, C., Cui, J., Wang, B., Zhang, M. and Su, J. (2022) Exploring Multi-Stage Information Interactions for Multi-Source Neural Machine Translation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30, 562-570. [Google Scholar] [CrossRef
[50] 亢晓勉, 宗成庆. 基于篇章结构多任务学习的神经机器翻译[J]. 软件学报, 2022, 33(10): 3806-3818.
[51] Leong, C.K., Liu, X., Wong, D.F. and Chao, L.S. (2021) Exploiting Translation Model for Parallel Corpus Mining. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29, 2829-2839. [Google Scholar] [CrossRef
[52] Li, X., Li, G., Liu, L., Meng, M. and Shi, S. (2019) On the Word Alignment from Neural Machine Translation. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, 28 July-2 August 2019, 1293-1303. [Google Scholar] [CrossRef
[53] 邹翔, 朱俊国, 高盛祥, 余正涛, 杨福岸. 融入语言差异化特征的汉越神经机器翻译译文质量估计[J]. 小型微型计算机系统, 2022, 43(7): 1413-1418.
[54] Song, K., Wang, K., Yu, H., Zhang, Y., Huang, Z., Luo, W., Duan, X. and Zhang, M. (2020) Alignment-Enhanced Transformer for Constraining NMT with Pre-Specified Translations. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 8886-8893. [Google Scholar] [CrossRef
[55] Wang, W., Chen, G., Wang, H., Han, Y. and Chen, Y. (2022) Multilingual Sentence Transformer as a Multilingual Word Aligner. Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, December 2022, 2952-2963.
[56] Zhang, L. and Komachi, M. (2018) Neural Machine Translation of Logographic Languages Using Sub-character Level Information. Proceedings of the Third Conference on Machine Translation: Research Papers, Brussels, 31 October-1 Novermber 2018, 17-25. [Google Scholar] [CrossRef
[57] Zhang, J. and Matsumoto, T. (2019) Character Decomposition for Japanese-Chinese Character-Level Neural Machine Translation. Proceedings of the 2019 International Conference on Asian Language Processing (IALP), Shanghai, 15-17 November 2019, 35-40. [Google Scholar] [CrossRef
[58] Zhang, L. and Komachi, M. (2019) Chinese-Japanese Unsupervised Neural Machine Translation Using Sub-character Level Information. The 33rd Pacific Asia Conference on Language, Information and Computation (PACLIC 33), Hakodate, 13-15 September 2019, 309-315.
[59] Du, J. and Way, A. (2017) Pinyin as Subword Unit for Chinese-Sourced Neural Machine Translation. Proceedings of the 2017 International Conference on Artificial Intelligence and Computer Science, Las Vegas, 14-16 December 2017, 89-101.
[60] Tan, M., Hu, Y., Nikolov, N.I. and Hahnloser, R. (2018) Wubi2en: Character-Level Chinese-English Translation through ASCII Encoding. Proceedings of the Third Conference on Machine Translation: Research Papers, Brussels, 31 October-1 November 2018, 10-16.
[61] Zhang, W., Lin, F., Wang, X., Liang, Z. and Huang, Z. (2019) SubCharacter Chinese-English Neural Machine Translation with Wubi Encoding.
https://arxiv.org/abs/1911.02737
[62] Halpern, J. (2018) Very Large-Scale Lexical Resources to Enhance Chinese and Japanese Machine Translation. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, 7-12 May 2018, 857-861.
[63] Saunders, D., Feely, W. and Byrne, B. (2020) Inference-Only Sub-Character Decomposition Improves Translation of Unseen Logographic Characters. Proceedings of the 7th Workshop on Asian Translation, Suzhou, December 2020, 170-177.
[64] 中澤敏明, Chu, C., 黒橋禎夫. 日中共通漢字の整理とこれを利用した日中機械翻訳の高度化[EB/OL]. Japio Year Book, 258-261.
https://cir.nii.ac.jp/crid/1523669555917032960, 2023-05-30.
[65] ト朝暉. 日中機械翻訳に関する研究~とりたて表現, 否定表現の翻訳規則を中心に~[D]: [博士学位论文]. 岐阜: 岐阜大学, 2004.
[66] 罗雯涛. 关于日中神经网络机器翻译中的词汇问题的探讨[J]. 计算机科学与应用, 2020(3): 387-397.