Stiefel流形上非光滑优化的一种带外推的可变度量邻近梯度算法
A Variable Metric Proximal Gradient Method with Extrapolation for Nonsmooth Optimization over the Stiefel Manifold
摘要: 本文针对Stiefel流形上一类目标函数为光滑损失函数与非光滑函数之和的非凸优化问题，提出了一种基于收缩的可变度量惯性邻近梯度算法。所提出的算法在已有的加速黎曼邻近梯度算法基础上，引入了对角Barzilai-Borwein类步长策略，该策略可以更好的捕获问题的局部几何信息，进一步加速算法的收敛。理论上，证明了算法全局收敛到稳定点。最后，本文给出了稀疏主成分分析问题的数值结果，验证了该方法的有效性。
Abstract: In this paper, we propose a retraction based variable metric inertial proximal gradient method for solving a class nonconvex optimization problem over the Stiefel manifold whose objective function is the summation of a smooth cost function and a nonsmooth function. Based on existing inertial Riemannian proximal gradient method, the proposed method introduces a metric changing called diagonal Barzilai-Borwein step-size strategy at each iteration, which can better capture the local geometric of this class problem and accelerate the convergence of the algorithm. Theoretically, we show that the proposed method globally converges to a stationary point. Numerical results on solving sparse PCA problem is reported to demonstrate the efficiency of the proposed method.
文章引用：张金超. Stiefel流形上非光滑优化的一种带外推的可变度量邻近梯度算法[J]. 应用数学进展, 2022, 11(3): 1107-1115. https://doi.org/10.12677/AAM.2022.113119

参考文献

[1] Erichson, N.B., Zheng, P., Manohar, K., Brunton, S.L., Kutz, J.N. and Aravkin, A.Y. (2020) Sparse Principal Compo-nent Analysis via Variable Projection. SIAM Journal on Applied Mathematics, 80, 977-1002. [Google Scholar] [CrossRef] 
[2] Barekat, F. (2014) On the Consistency of Compressed Modes for Variational Problems Associated with the Schrödinger Operator. SIAM Journal on Mathematical Analysis, 46, 3568-3577. [Google Scholar] [CrossRef] 
[3] Zhang, Y., Lau, Y., Kuo, H.-W., Cheung, S., Pasupathy, A. and Wright, J. (2017) On the Global Geometry Ofsphere-Constrained Sparse Blind Deconvolution. 2017 IEEE Con-ference on Computer Vision and Pattern Recognition, Honolulu, 21-26 July 2017, 4381-4389. https://par.nsf.gov/biblio/10203504 [Google Scholar] [CrossRef] 
[4] Tang, J. and Liu, H. (2012) Unsupervised Feature Selection for Linked Social Mediadata. Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Beijing, 12-16 August 2012, 904-912. https://dl.acm.org/doi/10.1145/2339530.2339673 [Google Scholar] [CrossRef] 
[5] Absil, P.-A. and Hosseini, S. (2019) A Collection of Nonsmooth Riemannian Optimization Problems. In: Hosseini, S., Mordukhovich, B. and Uschmajew, A., Eds., Nonsmooth Optimization and Its Applications, International Series of Numerical Mathematics, Vol. 170, Birkhäuser, Cham, 1-15. [Google Scholar] [CrossRef] 
[6] Hu J, Jiang, B., Liu, X. and Wen, Z.W. (2016) A Note on Semidenite Programming Relaxations for Polynomialoptimization over a Single Sphere. Science China Mathematics, 59, 1543-1560. [Google Scholar] [CrossRef] 
[7] Li, X., Chen, S., Deng, Z., Qu, Q., Zhu, Z. and So, A.M.-C. (2021) Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods. SIAM Journal on Optimization, 31, 1605-1634. https://epubs.siam.org/doi/10.1137/20M1321000 [Google Scholar] [CrossRef] 
[8] Hosseini, S. and Uschmajew, A. (2017) A Riemannian Gradient Sampling Algorithm for Nonsmooth Optimization on Manifolds. SIAM Journal on Optimization, 27, 173-189. https://epubs.siam.org/doi/abs/10.1137/16M1069298 [Google Scholar] [CrossRef] 
[9] Lai, R. and Osher, S. (2014) A Splitting Method for Orthogonality Constrained Problems. Journal of Scientific Computing, 58, 431-449. [Google Scholar] [CrossRef] 
[10] Chen, W., Ji, H. and You, Y. (2016) An Augmented Lagrangian Method for -Regularized Optimization Problems with Orthogonality Constraints. SIAM Journal on Scientific Com-puting, 38, B570-B592. https://epubs.siam.org/doi/abs/10.1137/140988875 [Google Scholar] [CrossRef] 
[11] Ferreira, O.P. and Oliveira, P.R. (2002) Proximal Point Algorithm on Riemannian Manifold. Optimization, 51, 257-270. [Google Scholar] [CrossRef] 
[12] Bento, G.C., Cruz Neto, J.X. and Oliveira, P.R. (2016) A New Approach to the Proximal Pointmethod: Convergence on General Riemannian Manifolds. Journal of Optimization Theory and Applications, 168, 743-755. [Google Scholar] [CrossRef] 
[13] Chen, S., Ma, S., Man-Cho So, A. and Zhang, T. (2020) Proxi-mal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold. SIAM Journal on Optimization, 30, 210-239. https://epubs.siam.org/doi/abs/10.1137/18M122457X [Google Scholar] [CrossRef] 
[14] Xiao, X., Li, Y., Wen, Z. and Zhang, L. (2018) A Regularized Semi-Smooth Newton Method with Projection Steps for Composite Convex Programs. Journal of Scientific Computing, 76, 364-389. [Google Scholar] [CrossRef] 
[15] Huang, W. and Wei, K. (2022) An Extension of Fast Iterative Shrinkage-Thresholding Algorithm to Riemannian Optimization for Sparse Principal Component Analysis. Numerical Linear Algebra with Applications, 29, Article No. e2409. [Google Scholar] [CrossRef] 
[16] Huang, W. and Wei, K. (2021) Riemannian Proximal Gradient Methods. Mathematical Programming, 1-43. [Google Scholar] [CrossRef] 
[17] Bonettini, S., Porta, F. and Ruggiero, V. (2016) A Variable Metric Forward-Backward Method with Extrapolation. SIAM Journal on Scientific Computing, 38, A2558-A2584. https://epubs.siam.org/doi/10.1137/15M1025098 [Google Scholar] [CrossRef] 
[18] Park, Y., Dhar, S., Boyd, S. and Shah, M. (2020) Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize. 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, Barcelona, 4-8 May 2020, 3597-3601. https://ieeexplore.ieee.org/document/9054193 [Google Scholar] [CrossRef] 

为你推荐