基于正则化风险函数模型的束方法
The Bundle Methods Based on the Regularized Risk Minimization Model
摘要:
本文在机器学习(ML)的正则化风险函数模型(SRM)的基础上,结合求解非光滑函数的束方法,提出了一种求解经验风险函数模型的算法,用割平面模型去近似目标函数,用非精确线搜索去获得步长,在适当的假设下,分析了算法的全局收敛和收敛速度。
Abstract:
Based on the regularized risk function model (SRM) of machine learning (ML), this paper combines the bundle method for solving non-smooth functions, presents an algorithm for solving empirical risk function model. The objective function is approximated by the cut-plane model, and the step size is obtained by the inexact line search. Under appropriate assumptions, the global convergence and convergence speed of the algorithm are analyzed.
参考文献
|
[1]
|
周志华. 机器学习[M]. 北京: 清华大学出版社, 2016.
|
|
[2]
|
李航. 统计机器学习[M]. 北京: 清华大学出版社, 2012.
|
|
[3]
|
袁亚湘, 孙文瑜. 最优化理论与方法[M]. 武汉: 科学出版社, 2002.
|
|
[4]
|
Kelley, J.E. (1960) The Cutting-Plane Method for Solving Convex Programs. Journal of the Society for Industrial & Applied Mathematics, 8, 703-712. [Google Scholar] [CrossRef]
|
|
[5]
|
陈韵梅, 张维. 基于近似一阶信息的加速的Bundle Level算法[J]. 中国科学: 数学, 2017(10): 1119-1142.
|
|
[6]
|
Teo, C.H., Viswanathan, S.V.N., Smola, A.J. and Le, Q. (2010) Bundle Methods for Regularized Risk Minimization, Journal of Machine Learning Research, 11, 311-365.
|
|
[7]
|
Shalev-Shwartz and Singer, Y. (2006) Online Learning Meets Optimization in the Dual. In: Simon, H.U. and Lugosi, G., Eds., Computational Learning Theory, Springer, Berlin, 423-437. [Google Scholar] [CrossRef]
|
|
[8]
|
Abe, N., Takeuchi, J. and Warmuth, M.K. (2001) Polynomial Learn Ability of Sto-chastic Rules with Respect to the KL-Divergence and Quadratic Distance. IEICE Transactions on Information and Systems, 84, 299-316.
|