一类解决可分离优化问题动力系统的渐进行为分析
A Gradual Asymptotic Behavior Analysis of Dynamical Systems for Solving Separable Optimization Problems
DOI: 10.12677/AAM.2024.1312504, PDF,   
作者: 董兴港:湖南大学电气与信息工程学院,湖南 长沙
关键词: 惯性动力系统可分离优化快速收敛性Inertial Dynamical Systems Separable Optimization Fast Convergence
摘要: 本文提出了一类关于二阶原始变量和一阶对偶变量组合的惯性原对偶动力系统,利用该惯性原对 偶动力系统来解决用来解决一类可分离的优化问题。 我们的方法与传统的优化算法不同,该方法 利用连续性的方程来解决最优化问题,在具体的应用过程当中,能够使最目标函数快速且渐近的 收敛到最优解。 我们主要得到两个重要理论:一方面在解决凸问题时,得到了该系统解的存在唯 一性;另一方面,基于Lyapunov分析方法的应用,在得到目标函数收敛到最优解的同时,还得到 了其收敛的速率为O(1/t2), 这也是己知的关于动力系统最好的收敛性结果。
Abstract: This paper introduces a class of inertial primal-dual dynamical systems involving second-order primal variables and first-order dual variables, which are used to solve a class of separable optimization problems. Our approach differs from traditional op- timization algorithms in that it utilizes continuous equations to solve optimization problems, enabling the objective function to converge quickly and asymptotically to the optimal solution in practical applications. We have obtained two important the- oretical results: on the one hand, the system has a unique solution; on the other hand, based on the application of the Lyapunov analysis method, we have not only obtained the convergence of the objective function to the optimal solution but also achieved a convergence rate of O(1/t2), which is also the best known convergence result for dynamical systems.
文章引用:董兴港. 一类解决可分离优化问题动力系统的渐进行为分析[J]. 应用数学进展, 2024, 13(12): 5225-5236. https://doi.org/10.12677/AAM.2024.1312504

参考文献

[1] Polyak, B.T. (1964) Some Methods of Speeding up the Convergence of Iteration Methods. USSR Computational Mathematics and Mathematical Physics, 4, 1-17.
https://doi.org/10.1016/0041-5553(64)90137-5
[2] Polyak, B.T. (1987) Introduction to Optimization, Optimization Software. In: Translations Series in Mathematics and Engineering. Publications Division Inc.
[3] Su, W.J., Boyd, S. and Cand´es, E. (2016) A Differential Equation for Modeling Nesterov’s Accelerated Gradient Method: Theory and Insights. Journal of Machine Learning Research, 17, 5312-5354.
[4] Nesterov,Y.E. (1983) A Method for Solving the Convex Programming Problem with Conver- gence Rate O(1/k2). Doklady Akademii Nauk, 269, 543-547.
[5] Zeng, X., Lei, J. and Chen, J. (2023) Dynamical Primal-Dual Nesterov Accelerated Method and Its Application to Network Optimization. IEEE Transactions on Automatic Control, 68, 1760-1767.
https://doi.org/10.1109/tac.2022.3152720
[6] He, X., Hu, R. and Fang, Y.P. (2022) “Second-Order Primal” + “First-Order Dual” Dynamical Systems with Time Scaling for Linear Equality Constrained Convex Optimization Problems. IEEE Transactions on Automatic Control, 67, 4377-4383.
[7] Zhu, T., Hu, R. and Fang, Y. (2024) Tikhonov Regularized Second-Order Plus First-Order Primal-Dual Dynamical Systems with Asymptotically Vanishing Damping for Linear Equality Constrained Convex Optimization Problems. Optimization, 1-28.
https://doi.org/10.1080/02331934.2024.2407515
[8] He, X., Hu, R. and Fang, Y.P. (2021) Convergence Rates of Inertial Primal-Dual Dynam- ical Methods for Separable Convex Optimization Problems. SIAM Journal on Control and Optimization, 59, 3278-3301.
https://doi.org/10.1137/20m1355379
[9] He, X., Tian, F., Li, A. and Fang, Y. (2023) Convergence Rates of Mixed Primal-Dual Dy- namical Systems with Hessian Driven Damping. Optimization, 1-26.
https://doi.org/10.1080/02331934.2023.2253813
[10] Hulett, D.A. and Nguyen, D. (2023) Time Rescaling of a Primal-Dual Dynamical System with Asymptotically Vanishing Damping. Applied Mathematics Optimization, 88, Article No. 27.
https://doi.org/10.1007/s00245-023-09999-9
[11] Bach, F. (2013) Learning with Submodular Functions: A Convex Optimization Perspective. Foundations and Trends in Machine Learning, 6, 145-373.
https://doi.org/10.1561/2200000039
[12] Wright, J., Ganesh, A. and Rao, S. (2009) Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization.
https://api.semanticscholar.org/CorpusID:212563318
[13] Condat, L. (2012) A Primal-Dual Splitting Method for Convex Optimization Involving Lips- chitzian, Proximable and Linear Composite Terms. Journal of Optimization Theory and Ap- plications, 158, 460-479.
https://doi.org/10.1007/s10957-012-0245-9
[14] Attouch, H., Chbani, Z. and Riahi, H. (2018) Combining Fast Inertial Dynamics for Convex Optimization with Tikhonov Regularization. Journal of Mathematical Analysis and Applica- tions, 457, 1065-1094.
https://doi.org/10.1016/j.jmaa.2016.12.017