求解具有等式约束的随机优化问题的二阶微分方程方法
Second Order Differential Equation Algorithm for Solving Stochastic Optimization Problems with Equality Constraints
摘要: 针对具有等式约束的随机优化问题，本文首先考虑基于SAA方法获取原始问题目标函数的SAA问题，并利用惩罚函数法将其转化为一般无约束凸优化问题。接下来建立二阶微分方程算法对所得到的问题进行求解。本文还证明了当算法的参数满足一定条件时，二阶微分方程算法具有良好的收敛性，其收敛率为，最后给出具体的数值算例验证算法的可行性。
Abstract: For the stochastic optimization problem with equality constraint, this paper first considers the SAA problem of obtaining the objective function of the original problem based on the SAA method, and uses the penalty function method to transform it into a general unconstrained convex optimization problem. Next, a second-order differential equation algorithm is established to solve the obtained problem. This paper also proves that when the parameters of the algorithm meet certain conditions, the second-order differential equation algorithm has good convergence, and its convergence rate is , and finally a specific numerical example is given to verify the feasibility of the algorithm.
文章引用：吕琪楠, 孙菊贺, 王莉, 谢红俭. 求解具有等式约束的随机优化问题的二阶微分方程方法[J]. 应用数学进展, 2023, 12(10): 4529-4537. https://doi.org/10.12677/AAM.2023.1210444

参考文献

[1] Ferguson, A.R. and Dantzig, G.B. (1956) The Allocation of Aircraft to Routes—An Example of Linear Programming under Uncertain Demand. Management Science, 3, 45-73. [Google Scholar] [CrossRef] 
[2] Dantzig, G.B. (2004) Linear Programming under Uncertainty. Management Science, 50, 1764-1769. [Google Scholar] [CrossRef] 
[3] Kiefer, J. and Wolfowitz, J. (1952) Stochastic Estimation of the Maximum of a Regression Function. The Annals of Mathematical Statistics, 23, 462-466. [Google Scholar] [CrossRef] 
[4] Spall, J.C. (1992) Multivariate Stochastic Approximation Using a Simultaneous Perturbation Gradient Approximation. IEEE Transactions on Automatic Control, 37, 332-341. [Google Scholar] [CrossRef] 
[5] Chen, R., Menickelly, M. and Scheinberg, K. (2018) Stochastic Optimization Using a Trust-Region Method and Random Models. Mathematical Programming, 169, 447-487. [Google Scholar] [CrossRef] 
[6] Billups, S.C., Graf, P. and Larson, J. (2013) Derivative-Free Optimization of Expensive Functions with Computational Error Using Weighted Regression. SIAM Journal on Optimization, 23, 27-53. [Google Scholar] [CrossRef] 
[7] Guo, J.H., Liu, H.L. and Xiao, X.T. (2020) A Unified Analysis of Stochastic Gradient-Free Frank-Wolfe Methods. International Transactions in Operational Research, 29, 63-86. [Google Scholar] [CrossRef] 
[8] Curtis, F.E., Scheinberg, K. and Shi, R. (2017) A Stochastic Trust Region Algorithm. Industrial and Systems Engineering, 1-16.
[9] Schmidt, M., Roux, N.L. and Bach, F. (2017) Minimizing Finite Sums with the Stochastic Average Gradient. Mathematical Programming, 162, 83-112. [Google Scholar] [CrossRef] 
[10] Nie, Y.Y. (2013) Convergence Theory of a SAA Method for Min-Max Stochastic Optimization Problems. Applied Mechanics and Materials, 303-306, 1319-1322. [Google Scholar] [CrossRef] 
[11] Ghadimi, S. and Lan, G. (2016) Accelerated Gradient Methods for Nonconvex Nonlinear and Stochastic Programming. Mathematical Programming, 156, 59-99. [Google Scholar] [CrossRef] 
[12] Chen, X.J., Qi, L. and Womersley, R.S. (1995) Newton’s Method for Quadratic Stochastic Programs with Recourse. Journal of Computational and Applied Mathematics, 60, 29-46. [Google Scholar] [CrossRef] 
[13] Nemirovski, A., Juditsky, A., Lan, G., et al. (2009) Robust Stochastic Approximation Approach to Stochastic Programming. SIAM Journal on Optimization, 19, 1574-1609. [Google Scholar] [CrossRef] 
[14] Qu, X. and Bian, W. (2022) Fast Inertial Dynamic Algorithm with Smoothing Method for Nonsmooth Convex Optimization. Computational Optimization and Applications, 3, 287-317. [Google Scholar] [CrossRef] 
[15] Polyak, B.T. and Juditsky, A.B. (1992) Acceleration of Stochastic Approximation by Averaging. SIAM Journal on Control and Optimization, 30, 838-855. [Google Scholar] [CrossRef] 

为你推荐