基于双层规划的代价敏感SVM超参数优化方法

Hyperparameter Optimization of Cost-Sensitive SVM via Bilevel Optimization

摘要: 针对类别不均衡且漏判代价更高的二分类问题，提出一种基于双层规划的代价敏感SVM超参数优化方法。该方法在训练集上采用RN策略修正类别权重，在验证集上以风险最小化学习惩罚系数，并利用forward-mode超梯度递推完成求解。UCI Breast Cancer Wisconsin数据集实验表明，raw30特征空间下召回率由92.86%提高到95.24%，总代价由32降至22；在统一100次评估预算下，相比anneal可更快达到更低验证损失。结果表明，该方法能有效降低高代价漏判风险，但RN增益受特征表达方式影响。

Abstract: For binary classification tasks with class imbalance and higher false-negative cost, a cost-sensitive SVM hyperparameter optimization method based on bilevel optimization is proposed. The method applies the RN strategy on the training set to revise class weights, learns the penalty parameter by minimizing risk on the validation set, and solves the problem using forward-mode hypergradient recursion. Experiments on the UCI Breast Cancer Wisconsin dataset show that, in the raw30 feature space, recall improves from 92.86% to 95.24%, while the total cost decreases from 32 to 22. Under a unified budget of 100 evaluations, the proposed method reaches a lower validation loss faster than anneal. The results indicate that the method can effectively reduce high-cost false-negative risk, although the gain brought by RN depends on the feature representation.