以得分优化为目标的损失函数

Loss Function Aimed at Score Optimization

摘要: 在机器学习的二分类任务中，损失函数对模型性能至关重要，但传统损失函数常难以直接优化准确率、F1得分等关键评估指标。针对这一问题，本研究融合两种创新方法，提出基于近似Heaviside函数的得分导向损失函数(SOLH)。研究通过用梯度友好函数近似Heaviside阶跃函数，并将分类阈值视为随机变量，对近似阶跃函数取期望，实现评估指标的端到端可微，进而构建损失函数。理论分析表明，该损失函数满足Lipschitz连续性，适合优化。在艾滋病临床试验组数据集上的实验结果显示，以F1得分优化为目标时，SOLH显著优于基线方法；在优化准确率方面，虽略逊于二元交叉熵损失函数，但仍优于其他基线方法。本研究成功整合两种前沿思路，搭建起训练与评估间的桥梁，不仅为损失函数性质提供理论依据，更通过实验验证其提升模型性能的有效性，为机器学习领域损失函数设计研究与实践应用开辟了新方向。

Abstract: In binary classification tasks of machine learning, loss functions are pivotal in determining model performance. Nevertheless, traditional loss functions frequently fail to directly optimize critical evaluation metrics like accuracy and F1-score. Aiming at this problem, this study integrates two innovative approaches and proposes the Score-Oriented Loss with approximate Heaviside function (SOLH). The research approximates the Heaviside step function with a gradient-friendly function, regards the classification threshold as a random variable, and calculates the expectation of the approximate step function, thus achieving end-to-end differentiability of evaluation metrics and constructing the loss function. Theoretical analysis indicates that this loss function meets the Lipschitz continuity condition, rendering it suitable for optimization. Experiments conducted on the AIDS Clinical Trials Group dataset show that when optimizing for the F1-score, SOLH outperforms baseline methods significantly. When it comes to accuracy optimization, although SOLH lags slightly behind the binary cross-entropy loss function, it still surpasses other baseline methods. This study successfully combines two cutting-edge concepts, bridging the gap between training and evaluation. It not only offers a theoretical foundation for the properties of loss functions but also validates the effectiveness of improving model performance through experiments, opening up new avenues for the research and practical application of loss function design in the field of machine learning.