一种面向资源受限环境的高吞吐量、置信度门控的细粒度文本分类框架

A Confidence-Gated Framework for High-Throughput Fine-Grained Text Classification in Resource-Constrained Environments

摘要: 针对通用大语言模型在资源受限环境下处理大规模细粒度文本分类任务时面临的效率瓶颈与幻觉问题，本文提出了一种名为DeepConf-Verify (DCV)的高性能框架。该框架首先通过领域微调将小参数量模型的准确率基线从不足30%显著提升至90%以上；进而引入双阈值动态置信度门控机制，利用词元级置信度轨迹实时监控生成过程，实现对“困惑”样本的立即熔断和对高确信样本的快速通行；最后，对处于临界置信度区间的样本执行双模型一致性验证以消除尾部风险。实验结果表明，在单张NVIDIA A100 GPU受限条件下，DCV框架在保持95.2%企业级准确率的同时，相比原有的通用大模型系统实现了超过1200%的吞吐量提升(达60.2条/秒)，相比同参数量的单一微调模型亦有24%的效率优化。系统成功支持日处理超过500万条评论数据，并将人工审核率控制在4.5%以内。本研究为在低资源环境下构建高吞吐、高可靠的垂直领域AI系统提供了有效的理论与实践范式。

Abstract: To address the efficiency bottlenecks and hallucination issues faced by general-purpose Large Language Models (LLMs) in handling large-scale, fine-grained text classification tasks within resource-constrained environments, this paper proposes a high-performance framework named DeepConf-Verify (DCV). Building upon domain-specific fine-tuning, which elevates the accuracy baseline of small-parameter models from under 30% to over 90%, the framework integrates a Dual-Threshold Dynamic Confidence Gating mechanism. This mechanism utilizes token-level confidence trajectories to monitor the generation process in real-time, executing an immediate “Panic Exit” for “confused” samples and a “Fast Pass” for high-confidence samples. Furthermore, a Dual-Model Verification protocol is employed to enforce consensus on samples within critical confidence intervals, thereby mitigating tail risks. Experimental results on a single NVIDIA A100 GPU demonstrate that DCV achieves an enterprise-grade accuracy of 95.2%. Notably, it boosts throughput by over 1200% (reaching 60.2 comments/sec) compared to the original general-purpose LLM system, and achieves a 24% efficiency optimization compared to a single fine-tuned model of equivalent parameter size. The system successfully scales to process over 5 million comments daily while keeping the manual audit rate within 4.5%. This study provides a robust theoretical and practical paradigm for constructing high-throughput and reliable vertical-domain AI systems in low-resource settings.