MCP正则优化问题的高效二阶算法
Efficient Second-Order Algorithm for MCP Regular Optimization Problems
摘要: Minimax Concave Penalty (MCP)正则优化问题在诸多科学领域有着广泛的应用，例如：机器学习、信号、图像恢复以及逻辑回归等问题。本文基于MCP正则项研究了一种二阶加速优化算法，该算法的主要思想是在于如何得到一个使得目标快速下降的方向，主要的方法是设计对偶半光滑牛顿法求解子问题。我们基于所提出的模型提出新的算法。通过数值试验部分的对比验证了我们所提出的算法的有效性和高效性。
Abstract: The Minimax Concave Penalty (MCP) regularization optimization problem is widely used in many scientific fields, such as machine learning, signal restoration, image restoration and logistic regres-sion. This paper studies a second-order accelerated optimization algorithm based on MCP regulari-zation term. The main idea of the algorithm is how to get a direction that makes the target drop quickly. The main method is to design dual semi-smooth Newton method to solve the subproblem. We propose a new algorithm based on the proposed model. The effectiveness and efficiency of the proposed algorithm are verified by numerical experiments.
文章引用：张原浩. MCP正则优化问题的高效二阶算法[J]. 应用数学进展, 2022, 11(10): 7173-7184. https://doi.org/10.12677/AAM.2022.1110761

参考文献

[1] Chen, S.X., Ma, S.Q., Man-Cho Soand, A. and Zhang, T. (2020) Proximal Gradient Method for Nonsmooth Optimiza-tion over the Stiefel Manifold. SIAM Journal on Optimization, 30, 210-239. [Google Scholar] [CrossRef] 
[2] Jin, Z.F., Wan, Z.P., Jiao, Y.L. and Lu, X.L. (2016) An Alternating Di-rection Method with Continuation for Nonconvex Low Rank Minimization. Journal of Scientific Computing, 66, 849-869. [Google Scholar] [CrossRef] 
[3] Rakotomamonjy, A., Flamary, R. and Gasso, G. (2015) Dc Proximal Newton for Nonconvex Optimization Problems. IEEE Transactions on Neural Networks and Learning Sys-tems, 27, 636-647. [Google Scholar] [CrossRef] 
[4] Li, X.D., Sun, D.F. and Toh, K.C. (2018) A highly Efficient Semismooth Newton Augmented Lagrangian Method for Solving Lasso Problems. SIAM Journal on Optimization, 28, 433-458. [Google Scholar] [CrossRef] 
[5] Marquardt, D.W. (1963) An Algorithm for Least-Squares Es-timation of Nonlinear Parameters. Journal of the Society for Industrial and Applied Mathematics, 11, 431-441. [Google Scholar] [CrossRef] 
[6] Duchi, J.C. and Ruan, F. (2019) Solving (Most) of a Set of Quadratic Equali-ties: Composite Optimization for Robust Phase Retrieval. Information and Inference: A Journal of the IMA, 8, 471-529. [Google Scholar] [CrossRef] 
[7] Charisopoulos, V., Chen, Y.D., Davis, D., Díaz, M., Ding, L.J. and Drusvyatskiy, D. (2021) Low-Rank Matrix Recovery with Composite Optimization: Good Conditioning and Rapid Convergence. Foundations of Computational Mathematics, 21, 1505-1593. [Google Scholar] [CrossRef] 
[8] Wu, Z.M., Li, C.S., Li, M. and Lim, A. (2021) Inertial Proximal Gradient Methods with Bregman Regularization for a Class of Nonconvex Optimization Problems. Journal of Global Optimization, 79, 617-644. [Google Scholar] [CrossRef] 
[9] Wei, F.R. and Zhu, H.X. (2012) Group Coordinate Descent Al-gorithms for Nonconvex Penalized Regression. Computational Statistics & Data Analysis, 56, 316-326. [Google Scholar] [CrossRef] 
[10] Jiang, H., Zheng, W.H. and Dong, Y. (2021) Sparse and Robust Estimation with Ridge Minimax Concave Penalty. Information Sciences, 571, 154-174. [Google Scholar] [CrossRef] 
[11] Selesnick, I. (2017) Sparse Regularization via Convex Analysis. IEEE Transactions on Signal Processing, 65, 4481-4494. [Google Scholar] [CrossRef] 
[12] Shi, Y.Y., Huang, J., Jiao, Y.L. and Yang, Q.L. (2019) A Sem-ismooth Newton Algorithm for High-Dimensional Nonconvex Sparse Learning. IEEE Transactions on Neural Networks and Learning Systems, 31, 2993-3006. [Google Scholar] [CrossRef] 
[13] Wang, S.B., Chen, X.F., Dai, W.W., Selesnick, I.W., Cai, G. and Cowen, B. (2018) Vector Minimax Concave Penalty for Sparse Representation. Digital Signal Processing, 83, 165-179. [Google Scholar] [CrossRef] 
[14] Zhang, C.H. (2010) Nearly Unbiased Variable Selection under Minimax Concave Penalty. The Annals of Statistics, 38, 894-942. [Google Scholar] [CrossRef] 
[15] Xu, J.W. and Chao, M.T. (2021) An Inertial Bregman Generalized Alter-nating Direction Method of Multipliers for Nonconvex Optimization. Journal of Applied Mathematics and Computing, 68, 1-27. [Google Scholar] [CrossRef] 
[16] Cai, G.G., Wang, S.B, Chen, X.F., Ye, J.J. and Selesnick, I.W. (2020) Reweighted Generalized Minimax-Concave Sparse Regularization and Application in Machinery Fault Diagnosis. ISA Transactions, 105, 320-334. [Google Scholar] [CrossRef] [PubMed]
[17] Li, Z.P., Qiao, B.J., Wen, B., Li, Z.D. and Chen, X.F. (2021) Re-weighted Generalized Minimax-Concave Sparse Regularization for Duct Acoustic Mode Detection with Adaptive Threshold. Journal of Sound and Vibration, 506, Article ID: 116165. [Google Scholar] [CrossRef] 
[18] Li, H. and Lin, Z.C. (2015) Accelerated Proximal Gradient Methods for Nonconvex Programming. Proceedings of the 28th International Conference on Neural Information Processing Sys-tems, Vol. 1, Montreal, 7-12 December 2015, 379-387.
[19] Breheny, P. and Huang, J. (2011) Coordinate Descent Al-gorithms for Nonconvex Penalized Regression, with Applications to Biological Feature Selection. The Annals of Applied Statistics, 5, 232-253. [Google Scholar] [CrossRef] 
[20] Beck, A. (2017) First-order Methods in Optimization. Society for In-dustrial and Applied Mathematics, Philadelphia. [Google Scholar] [CrossRef] 
[21] Rockafellar, R.T. and Wets, R.J. B. (1998) Variational Analysis. Vol. 317, Springer Science & Business Media, Berlin, Heidelberg. [Google Scholar] [CrossRef] 
[22] Chang, C. and Lin, C.J. (2011) Libsvm: A Library for Support Vector Machines. ACM Transactions on Intelligent Systems and Technology, 2, Article No. 27. [Google Scholar] [CrossRef] 

为你推荐