稀疏加组稀疏优化问题的方向稳定点及其光滑化方法
Directional Stationary Points of Sparse plus Group Sparse Optimization Problems with Its Smoothing Methods
摘要: 本文研究稀疏加组稀疏优化问题的非凸松弛模型,其中惩罚项既含有稀疏惩罚,又含有组稀疏惩罚,对稀疏惩罚和组稀疏惩罚均采用折叠凹惩罚函数进行连续松弛,得到复合非光滑非凸优化模型。为刻画此模型的最优性条件,给出了其方向导数刻画和方向稳定点,分析了方向稳定点的特征及其局部最优性质。为计算模型的方向稳定点,构造了模型的光滑化逼近问题,并证明了光滑化问题的一阶稳定点收敛于模型的方向稳定点In this paper, we study the nonconvex relaxation model for the sparse plus group sparse optimiza-tion problem, in which the penalty term contains both sparse penalty and group sparse penalty. As continuous relaxations, the folded concave penalty functions are used to relax both sparse penalty and group sparse penalty, which results the compound nonsmooth and nonconvex optimization model. In order to characterize the optimality of the nonconvex relaxation problem, the directional derivative and the directional stationary point are introduced, and then the characteristics of the directional stationary points and its local optimality are analyzed. To calculate the directional sta-tionary points of the relaxation model, the smoothing approximation problem is constructed, and it is proved that the stationary points of the smoothing problem converge to the directional stationary point of the relaxation problem, which provides a theoretical guarantee for the calculation of the directional stationary point of the relaxation problem by using the smooth methods.,为使用光滑方法计算模型的方向稳定点提供了理论保证。
文章引用:苏妍妍, 彭定涛. 稀疏加组稀疏优化问题的方向稳定点及其光滑化方法[J]. 应用数学进展, 2022, 11(10): 7464-7477. https://doi.org/10.12677/AAM.2022.1110792

参考文献

[1] Bech, A. and Hallak, N. (2019) Optimization Problems Involving Group Sparsity Terms. Mathematical Programming, 178, 39-67. [Google Scholar] [CrossRef
[2] Hu, Y., Li, C., Meng, K., Qin, J. and Yang, X. (2017) Group Sparse Optimization via lp,q Regularization. Journal of Machine Learning Research, 18, 960-1011.
[3] Jiao, Y., Jin, B. and Lv, X. (2017) Group Sparse Recovery via the l0(l2) Penalty: Theory and Algorithm. IEEE Transactions on Signal Processing, 65, 998-1012. [Google Scholar] [CrossRef
[4] Li, W., Bian, W. and Toh, K.-C. (2022) Difference-of-Convex Algorithms for a Class of Sparse Group l0 Regularized Optimization Problems. SIAM Journal on Optimization, 32, 1614-1641. [Google Scholar] [CrossRef
[5] Pan, L. and Chen, X. (2021) Group Sparse Optimization for Images Recovery Using Capped Folded Concave Functions. SIAM Journal on Imaging Sciences, 14, 1-25. [Google Scholar] [CrossRef
[6] Natarajan, B.K. (1995) Sparse Approximate Solu-tions to Linear Systems. SIAM Journal on Computing, 24, 227-234. [Google Scholar] [CrossRef
[7] Li, X., Sun, D. and Toh, K.-C. (2018) A Highly Efficient Sem-ismooth Newton Augmented Lagrangian Method for Solving Lasso Problems. SIAM Journal on Optimization, 28, 433-458. [Google Scholar] [CrossRef
[8] Lin, M., Liu, Y. J., Sun, D. and Toh, K.C. (2019) Efficient Sparse Semismooth Newton Methods for the Clustered Lasso Problem. SIAM Journal on Optimization, 29, 2026-2052. [Google Scholar] [CrossRef
[9] Zhang, Y., Zhang, N., Sun, D. and Toh, K.-C. (2020) An Efficient Hes-sian-Based Algorithm for Solving Large-Scale Sparse Group Lasso Problems. Mathematical Programming, 179, 223-263. [Google Scholar] [CrossRef
[10] Candès, E.J., Wakin, M.B. and Boyd, S.P. (2008) En-hancing Sparsity by Reweighted l1 Minimization. Journal of Fourier Analysis and Applications, 14, 877-905. [Google Scholar] [CrossRef
[11] Fan, J. and Li, R. (2001) Variable Selection via Nonconvave Pe-nalized Likelihood and Its Oracle Properties. Journal of the American Statistical Association, 96, 1348-1360. [Google Scholar] [CrossRef
[12] Fan, J. and Li, R. (2006) Statistical Challenges with High Di-mensionality: Feature Selection in Knowledge Discovery. Proceedings of the International Congress of Mathematicians, 3, 595-622. [Google Scholar] [CrossRef
[13] Zhang, C.-H. (2010) Nearly Unbiased Variable Selection under Minimax Concave Penalty. Annals of Statistics, 38, 894-942. [Google Scholar] [CrossRef
[14] Thi, L.E. and Cheng, S.O. (2013) Learning Sparse Classifiers with Difference of Convex Functions Algorithms. Optimization Methods and Software, 28, 830-854. [Google Scholar] [CrossRef
[15] Tong, Z. (2010) Analy-sis of Multi-Stage Convex Relaxation for Sparse Regularization. Journal of Machine Learning Research, 11, 1081-1107.
[16] Nikolova, M., Ng, M.-K., Zhang, S. and Ching, W.-K. (2008) Efficient Reconstruction of Piecewise Constant Images Using Nonsmooth Nonconvex Minimization. SIAM Journal on Imaging Sciences, 1, 2-25. [Google Scholar] [CrossRef
[17] Bruckstein, A.M., Donoho, D.L. and Elad, M. (2009) From Sparse Solu-tions of Systems of Equations to Sparse Modeling of Signals and Images. SIAM Review, 51, 34-81. [Google Scholar] [CrossRef
[18] Knight, K. and Fu, W. (2000) Asymptotics for Lasso-Type Estimators. Annals of Statistics, 28, 1356-1378. [Google Scholar] [CrossRef
[19] Huang, J., Ma, S., Xie, H. and Zhang, C.H. (2009) A Group Bridge Approach for Variable Selection. Biometrika, 96, 339-355. [Google Scholar] [CrossRef] [PubMed]
[20] Ahn, M., Pang, J.-S. and Xin, J. (2017) Difference-of-Convex Learning: Directional Stationarity, Optimality, and Sparsity. SIAM Journal on Optimization, 27, 1637-1665. [Google Scholar] [CrossRef
[21] Chang, T.H., Hong, M. and Pang, J.-S. (2017) Local Minimizers and Second-Order Conditions in Composite Piecewise Programming via Directional Derivatives.
[22] Pang, J.-S., Razaviyayn, M. and Alvarado, A. (2017) Computing B-Stationary Points of Nonsmooth DC Programs. Mathematics of Operations Research, 42, 95-118. [Google Scholar] [CrossRef
[23] Rochafellar, R.T. and Wets, R.J.-B. (2009) Variational Analysis. 3rd Edition, Springer-Verlag, Berlin.
[24] Peng, D. and Chen, X. (2020) Computation of Second-Order Directional Station-ary Points for Group Sparse Optimization. Optimization Methods and Software, 35, 348-376. [Google Scholar] [CrossRef
[25] Chen, X. (2012) Smoothing Methods for Nonsmooth, No-vonvex Minimization. Mathematical Programming, 134, 71-99. [Google Scholar] [CrossRef
[26] Chen, X., Niu, L. and Yuan, Y. (2013) Optimality Conditions and a Smoothing Trust Region Newton Method for Non- Lipschitz Optimization. SIAM Journal on Optimization, 23, 1528-1552. [Google Scholar] [CrossRef
[27] Chen, X., Xu, F. and Ye, Y. (2010) Lower Bound Theory of Nonzero Entries in Solutions of l2-lp Minimization. SIAM Journal on Computing, 32, 2832-2852. [Google Scholar] [CrossRef