引入反馈机制的中文文本校对技术研究

Research on Chinese Proofreading Technology with Feed-Back Mechanism

摘要: 中文文本校对技术已取得了很大进展，然而目前很多技术研究依赖于深度学习，随着语言模型越来越复杂，训练成本迅速增加，导致落地应用较为困难。针对上述问题，本文提出了一种迭代式无监督文本自动校对技术，可同时纠正多字、少字、字序颠倒以及错别字等文本错误，并设计了反馈机制，可对校对错误结果进行反馈与实时修正。模型使用交叉位置融合算法定位错词索引，针对检测到的错词位置，采用并行多通道候选词构建策略得到候选词序列，并基于得分修正算法计算最优候选词。该方法在公开数据集SIGAHN和自构建数据集上进行了测试实验，纠正准确率和精度分别提升了6.39%和5.17%，高于Transformer等深度学习模型，且训练成本低，可作为文本自动校对技术普及应用的参考方案。

Abstract: Chinese proofreading technology has made great progress. However, at present, many technical studies rely on deep learning. As the language model becomes more and more complex, the training cost increases rapidly, resulting in difficulties in landing applications. In view of the above problems, this paper proposes an iterative unsupervised text automatic proofreading technology, which can correct text errors such as multi word, few word, reversed word order and wrong type words at the same time, and designs a feedback mechanism to feed back and correct the proofreading error results in real time. The model uses the cross position fusion algorithm to locate the wrong word index. For the detected wrong word position, it uses the parallel multi-channel candidate word construction strategy to get the candidate word sequence, and calculates the optimal candidate word based on the score correction algorithm. The method has been tested on the public data set sigahn and the self built data set, and the correction accuracy and precision have been improved by 6.39% and 5.17% respectively, which are higher than the transformer deep learning model, and the training cost is low. It can be used as a reference scheme for the popularization and application of automatic text proofreading technology.