基于Huber损失和Capped-L1正则的线性不等式约束稀疏优化问题研究
On Sparse Optimization Problems with Linear Inequality Constraints Based on Huber Loss and Capped-L1 Regularization
摘要: 对多元线性回归中回归系数的估计问题，本文考虑了基于Huber损失和线性不等式约束的稀疏优化模型。首先，给出了稀疏优化的原问题、基于Capped-L1正则的松弛问题和基于约束惩罚的无约束问题三种模型。其次，借助惩罚模型方向稳定点的下界性质，在一定条件下分析了三种模型全局最优解的等价性。最后，提出了光滑化惩罚算法，并证明了该算法的收敛性。本文为求解线性不等式约束稀疏优化问题提供了理论和方法基础。
Abstract: For the estimation of regression coefficients in multivariate linear regression, a sparse optimization model based on Huber loss and linear inequality constraints is considered in this paper. Firstly, three models of the original sparse optimization problem, the relaxation problem based on Capped-L1 regularization and the unconstrained problem based on the penalty of constraint are given. Secondly, by use of the lower bound property of the directional stationary point of the pe-nalized model, the equivalence of the global optimal solutions of the three models is analyzed under certain conditions. Finally, a smoothing penalty algorithm is proposed and its convergence is proved. This paper provides a theoretical and methodological basis for solving sparse optimization problems with linear inequality constraints.
文章引用：田梦达, 彭定涛, 张弦. 基于Huber损失和Capped-L1正则的线性不等式约束稀疏优化问题研究[J]. 理论数学, 2022, 12(11): 2021-2032. https://doi.org/10.12677/PM.2022.1211219

参考文献

[1] Natarajan, B. (1995) Sparse Approximate Solutions to Linear Systems. SIAM Journal on Computing, 24, 227-234. [Google Scholar] [CrossRef] 
[2] Donoho, D. (2006) Compressed Sensing. IEEE Transactions on Information Theory, 52, 1289-1306. [Google Scholar] [CrossRef] 
[3] Candès, E., Romberg, J. and Tao, T. (2006) Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information. IEEE Transactions on Infor-mation Theory, 52, 489-509. [Google Scholar] [CrossRef] 
[4] Tibshirani, R. (1996) Regression Shrinkage and Selection via the LASSO. Journal of the Royal Statistical Society: Series B (Methodological), 58, 267-288. [Google Scholar] [CrossRef] 
[5] Fan, J. and Li, R. (2001) Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties. Journal of the American Statistical Association, 96, 1348-1360. [Google Scholar] [CrossRef] 
[6] Zhang, C. (2010) Nearly Unbiased Variable Se-lection under Minimax Concave Penalty. Annals of Statistics, 38, 894-942. [Google Scholar] [CrossRef] 
[7] Ong, C. and An, L. (2013) Learning Sparse Classifiers with Difference of Convex Functions Algorithms. Optimization Methods and Software, 28, 830-854. [Google Scholar] [CrossRef] 
[8] Peleg, D. and Meir, R. (2008) A Bilinear Formulation for Vector Sparsity Optimization. Signal Processing, 88, 375-389. [Google Scholar] [CrossRef] 
[9] Thi, H., Dinh, T., Le, H. and Vo, X. (2015) DC Approximation Approaches for Sparse Optimization. European Journal of Operational Research, 244, 26-46. [Google Scholar] [CrossRef] 
[10] Zhang, T. (2013) Multi-Stage Convex Relaxation for Feature Se-lection. Bernoulli, 19, 2277-2293. [Google Scholar] [CrossRef] 
[11] Bian, W. and Chen, X. (2017) Optimality and Complexity for Constrained Optimization Problems with Nonconvex Regularization. Mathematics of Operations Research, 42, 1063-1084. [Google Scholar] [CrossRef] 
[12] Chartrand, R. and Staneva, V. (2008) Restricted Isometry Properties and Nonconvex Compressive Sensing. Inverse Problems, 24, 1-14. [Google Scholar] [CrossRef] 
[13] Huang, J., Horowitz, J. and Ma. S. (2008) Asymptotic Properties of Bridge Estimators in Sparse High-Dimensional Regression Models. Annals of Statistics, 36, 587-613. [Google Scholar] [CrossRef] 
[14] Ahn, M., Pang, J. and Xin, J. (2017) Difference-of-Convex Learning: Directional Stationarity, Optimality, and Sparsity. SIAM Journal on Optimization, 27, 1637-1655. [Google Scholar] [CrossRef] 
[15] An, L. and Tao, P. (2005) The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems. Annals of Op-erations Research, 133, 23-46. [Google Scholar] [CrossRef] 
[16] Pang, J., Razaviyayn, M. and Alvarado, A. (2017) Computing B-Stationary Points of Nonsmooth DC Programs. Mathematics of Operations Research, 42, 95-118. [Google Scholar] [CrossRef] 
[17] Chen, X., Niu, L. and Yuan, Y. (2013) Optimality Conditions and Smoothing Trust Region Newton Method for Non-Lipschitz Optimization. SIAM Journal on Optimization, 23, 1528-1552. [Google Scholar] [CrossRef] 
[18] Candès, E., Walkin, M. and Boyd, S. (2008) Enhancing Sparsity by Reweighted Minimization. Journal of Fourier Analysis and Applications, 14, 877-905. [Google Scholar] [CrossRef] 
[19] Bian, W. and Chen, X. (2020) A Smoothing Proximal Gradient Algorithm for Non-Smooth Convex Regression with Cardinality Penalty. SIAM Journal on Numerical Analysis, 58, 858-883. [Google Scholar] [CrossRef] 
[20] 罗孝敏, 彭定涛, 张弦. 基于MCP正则的最小一乘回归问题研究[J]. 系统科学与数学, 2021, 41(8): 2327-2337.
[21] 彭定涛, 唐琦, 张弦. 组稀疏优化问题精确连续Capped-L1松弛[J]. 数学学报, 2022, 65(2): 243-262.
[22] Pan, L. and Chen, X. (2021) Group Sparse Optimization for Images Recovery Using Capped Folded Concave Functions. SIAM Journal on Imaging Sciences, 14, 1-25. [Google Scholar] [CrossRef] 
[23] Zhang, X. and Peng, D. (2022) Solving Constrained Nonsmooth Group Sparse Optimization via Group Capped- Relaxation and Group Smoothing Proximal Gradient Algorithm. Computa-tional Optimization and Applications, 83, 801-804. [Google Scholar] [CrossRef] 
[24] Peng, D. and Chen, X. (2020) Computation of Second-Order Directional Stationary Points for Group Sparse Optimization. Op-timization Methods and Software, 35, 348-376. [Google Scholar] [CrossRef] 
[25] Rockafellar, R. and Wets, R. (2009) Variational Analysis. 3rd Edition, Springer-Verlag, Berlin.
[26] Chen, X., Lu, Z. and Pong, T. (2016) Penalty Methods for a Class of Non-Lipschitz Optimization Problems. SIAM Journal on Optimization, 26, 1465-1492. [Google Scholar] [CrossRef] 

为你推荐