BVTS：基于BERT增强的高质量语音合成模型

BVTS: A High-Quality Speech Synthesis Model Enhanced by BERT

摘要: 近年来，语音合成(Text-to-Speech, TTS)技术在端到端建模、音质优化等方面取得显著进展，合成语音的清晰度与流畅度大幅提升，但在逼近人类真实语音质感方面仍存挑战，主要瓶颈在韵律建模、语义理解适配方面欠缺。本文提出一种基于BERT (Bidirectional Encoder Representations from Transformers)模型增强的语音合成框架——BVTS (BERT-Integrated-VITS2)，模型以VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech)为框架，引入多模态文本编码器，在BERT特征嵌入的引导下，通过特征级融合语音学及韵律特征，同时采用双向可逆流模型与随机时长预测器，实现对语音节奏与语速的细粒度控制。在LJ Speech数据集与自制游戏数据集上的实验结果表明，相较于当前主流模型，BVTS的平均意见得分(MOS)整体提升明显，且字符错误率(CER)更低，此模型明显提升了合成语音的表现力、自然度与可懂度。

Abstract: In recent years, Text-to-Speech (TTS) technology has achieved significant progress in end-to-end modeling and sound quality optimization, with the clarity and fluency of synthesized speech improved substantially. However, challenges remain in approaching the texture of human real speech, and the main bottlenecks lie in the insufficient prosody modeling and semantic understanding adaptation. This paper proposes a BERT (Bidirectional Encoder Representations from Transformers)-enhanced speech synthesis framework named BVTS (BERT-Integrated-VITS2). Based on the VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech) framework, the model introduces a multimodal text encoder. Guided by BERT feature embedding, it fuses phonetic and prosodic features at the feature level. Meanwhile, it adopts a bidirectional reversible flow model and a random duration predictor to achieve fine-grained control over speech rhythm and speed. Experimental results on the LJ Speech dataset and the self-constructed game dataset show that compared with current mainstream models, BVTS achieves a significant overall improvement in Mean Opinion Score (MOS) and a lower Character Error Rate (CER). This model significantly enhances the expressiveness, naturalness and intelligibility of synthesized speech.