大语言模型在BEC高级写作评分中的效能对比研究——基于DeepSeek与豆包的实证分析

A Comparative Study on the Efficacy of Large Language Models in BEC Higher Writing Scoring—An Empirical Analysis of DeepSeek and Doubao

摘要: 为检验国产大语言模型在BEC高级商务英语作文评分中的实用性，本研究以DeepSeek-V3.2与豆包为对象，选取了90篇BEC高级作文，设置无提示词、仅提供评分标准、同时提供评分标准和人工打分范文三种提示词场景开展对比实验。结果显示，DeepSeek-V3.2整体评分准确性、稳定性均优于豆包，从作文类型来看，两款模型均对商务报告评分最准确，商务信函评分能力较弱。仅提供评分标准会降低模型评分效果，搭配人工范文可明显提升评分质量。两款模型均可用于作文初评，但与专业人工评分仍有差距，暂不能完全替代人工。本研究为商务英语写作的人机协同评分提供了参考。

Abstract: To examine the practicability of domestic large language models (LLMs) in the scoring of BEC Higher business English writing, this study selects DeepSeek-V3.2 and Doubao as research objects, and employs a dataset of 90 BEC Higher writing scripts to carry out controlled comparative experiments under three prompt scenarios: no prompt, only scoring criteria provided, and both scoring criteria and human-scored sample essays provided. The findings reveal that DeepSeek-V3.2 surpasses Doubao in both overall scoring accuracy and stability. In terms of writing genres, both models deliver the highest scoring accuracy for business report, while their scoring performance for business letter is relatively weaker. Providing only scoring criteria undermines the models’ scoring efficacy, whereas the combination of scoring criteria and annotated sample essays can markedly improve scoring quality. Although both LLMs are applicable to the preliminary evaluation of writing scripts, a distinct gap remains between LLMs scoring and human scoring, indicating that they cannot fully replace human raters for the time being. This research offers implications for the implementation of human-machine collaborative scoring in business English writing assessment.