一类混合稀疏组稀疏优化问题的邻近梯度算法
Proximal Gradient Algorithm for a Class of Mixed Sparse and Group Sparse Optimization Problems
摘要: 本文研究了一类混合稀疏组稀疏优化问题，其中损失函数为光滑凸函数，正则项为稀疏l1范数与组稀疏lα,p(α ≥ 1, p > 0)范数的组合。 首先，提出了邻近梯度算法求解此混合稀疏组稀疏优化问题。其次，分别讨论了凸(p ≥ 1)和非凸(0 < p < 1)两种情况下算法的收敛性。 最后，对于非凸问 题，在α = 1,时给出组合惩罚项邻近算子的闭式解。 本文结果为求解混合稀疏组稀疏优化问题提供了理论依据和可行途径。
Abstract: This paper investigates a class of mixed sparse and group sparse optimization problems in which the loss function is a smooth and convex function, and the regularization term is a combination of the sparse l1 norm and the group sparse lα,p norm (α ≥ 1, p > 0). Firstly, we propose a proximal gradient algorithm to solve this class of mixed sparse optimization problems. Secondly, we discuss the convergence properties in both convex (p ≥ 1) and non-convex (0 < p < 1) scenarios. For non-convex problems, closed-form solutions of the proximal operators to the mixed penalty can be obtained when α = 1, . The result provides a theoretical foundation and a feasible approach for solving mixed sparse and group sparse optimization problems.
文章引用：童兴华, 彭定涛, 张弦. 一类混合稀疏组稀疏优化问题的邻近梯度算法[J]. 运筹与模糊学, 2023, 13(6): 7598-7611. https://doi.org/10.12677/ORF.2023.136745

参考文献

[1] 刘浩洋, 户将, 李勇锋, 文再文. 最优化: 建模, 算法与理论[M]. 北京: 高等教育出版社, 2020.
[2] Li, W., Bian, W. and Toh, K.-C. (2022) Difference-of-Convex Algorithms for a Class of Sparse Group l0 Regularized Optimization Problems. SIAM Journal on Optimization, 32, 1614-1641.
https://doi.org/10.1137/21M1443455
[3] Jain, P., Rao, N. and Dhillon, I.S. (2016) Structured Sparse Regression via Greedy Hard Thresholding. Advances in Neural Information Processing Systems, 29, 1516-1524.
[4] Chen, X., Pan, L. and Xiu, N. (2023) Solution Sets of Three Sparse Optimization Problems for Multivariate Regression. Journal of Global Optimization, 87, 347-371.
https://doi.org/10.1007/s10898-021-01124-w
[5] Pan, L. and Chen, X. (2021) Group Sparse Optimization for Image Recovery Using Capped Folded Concave Functions. SIAM Journal on Imaging Sciences, 14, 1-25.
https://doi.org/10.1137/19M1304799
[6] Zhang, Y., Zhang, N., Sun, D. and Toh, K.C. (2020) An Efficient Hessian-Based Algorithm for Solving Large-Scale Sparse Group Lasso Problems. Mathematical Programming, 179, 223-263.
https://doi.org/10.1007/s10107-018-1329-6
[7] Zhang, Y., Zhang, N., Sun, D. and Toh, K.C. (2020) A Proximal Point Dual Newton Algorithm for Solving Group Graphical Lasso Problems. SIAM Journal on Optimization, 30, 2197-2220.
https://doi.org/10.1137/19M1267830
[8] Peng, D. and Chen, X. (2020) Computation of Second-Order Directional Stationary Points for Group Sparse Optimization. Optimization Methods and Software, 35, 348-376.
https://doi.org/10.1080/10556788.2019.1684492
[9] Luo, Z., Sun, D., Toh, K.C. and Xiu, N. (2019) Solving the OSCAR and SLOPE Models Using a Semismooth Newton-Based Augmented Lagrangian Method. Journal of Machine Learning Research, 20, 1-25.
[10] Chen, X. and Toint, P.L. (2021) High-Order Evaluation Complexity for Convexly-Constrained Optimization with Non-Lipschitzian Group Sparsity Terms. Mathematical Programming, 187, 47-78.
https://doi.org/10.1007/s10107-020-01470-9
[11] Beck, A. and Hallak, N. (2019) Optimization Problems Involving Group Sparsity Terms. Math- ematical Programming, 178, 39-67.
https://doi.org/10.1007/s10107-018-1277-1
[12] Zhang, Y., Wei, C. and Liu, X. (2022) Group Logistic Regression Models with lp,q Regulariza- tion. Mathematics, 10, Article 2227.
https://doi.org/10.3390/math10132227
[13] Cai, T.T., Zhang, A.R. and Zhou, Y.C. (2022) Sparse Group Lasso: Optimal Sample Complex- ity, Convergence Rate, and Statistical Inference. IEEE Transactions on Information Theory, 68, 5975-6002.
https://doi.org/10.1109/TIT.2022.3175455
[14] Beck, A. and Teboulle, M. (2009) A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems. SIAM Journal on Imaging Sciences, 2, 183-202.
https://doi.org/10.1137/080716542
[15] Bolte, J., Sabach, S. and Teboulle, M. (2014) Proximal Alternating Linearized Minimization for Nonconvex and Nonsmooth Problems. Mathematical Programming, 146, 459-494.
https://doi.org/10.1007/s10107-013-0701-9
[16] Hu, Y., Li, C., Meng, K., Qin, J. and Yang, X. (2017) Group Sparse Optimization via lp, q Regularization. The Journal of Machine Learning Research, 70, 960-1011.

为你推荐