深度学习与进化算法耦合下的最优多维随机控制问题
Solving Multi-Dimensional Optimal Stochastic Control Problems with Deep Learning and Evolution Algorithm
DOI: 10.12677/AAM.2022.113133, PDF,   
作者: 张智豪：上海理工大学，理学院，上海；徐 勉*：河南中烟工业有限责任公司洛阳卷烟厂，河南 洛阳
关键词: 偏微分方程；倒向随机微分方程；高维；深度学习；随机控制；进化算法；Partial Differential Equation； Backward Stochastic Differential Equation； High-Dimensional； Deep Learning； Stochastic Control； Evolution Algorithm
摘要: 受困于维数诅咒，能够求解高维偏微分方程(PDEs)的算法一直以来都极其有限。鄂维南和韩劼群在2017年提出的算法通过将未知解的梯度看作策略函数，利用深度学习可以较为有效的解决高维偏微分方程，但却无法解决带有真正策略函数的问题。本文提出了一种新算法，通过多层神经网络表示策略函数映射，将方程的解映射为适应度函数，把网络中的参数看作自变量，通过进化算法优化整个策略函数；同时配合鄂维南和韩劼群的算法求解问题。通过在Riccati方程和投资消费问题等的实际算例模拟下，表明了算法的准确性和实际意义。
Abstract: Because of the curse of dimensionality, developing efficient algorithms for solving high-dimensional partial differential equations (PDEs) has been an extremely difficult task. The algorithm which Weinan E and Jiequn Han proposed in 2017 views the gradient of the unknown solution as policy function, and through deep learning can effectively solve high-dimensional partial differential equations, but this method cannot deal with stochastic control problems with real policy function. We propose a new algorithm for solving this problem, which use multilayer neural network to represent the map of policy function and view the parameters in the neural network as independent variables. Then, we use the evolution algorithm to optimal the policy function. At the same time, we cooperate with Weinan E and Jiequn Han’s algorithm to solve this problem. Numerical results on 5-dimensional Riccati equation and 12-dimensional Investment and Consumption Problem suggest the accuracy and practical significance of the algorithm.
文章引用：张智豪, 徐勉. 深度学习与进化算法耦合下的最优多维随机控制问题[J]. 应用数学进展, 2022, 11(3): 1222-1241. https://doi.org/10.12677/AAM.2022.113133

参考文献

[1] Bellman, R. (1984) Dynamic Programming. Princeton University Press, Princeton.
[2] Powell, W.B. (2011) Approximate Dynamic Programming Solving the Curses of Dimensionality. Wiley, Princeton.
[3] Pardoux, E. and Peng, S. (1992) Backward Stochastic Differential Equations and Quasilinear Parabolic Partial Differential Equations. In: Rozovskii, B.L. Sowers, R.B. and Eds., Stochastic Partial Differential Equations and Their Applications, Springer-Verlag, Berlin/Heidelberg, Vol. 176, 200-217. [Google Scholar] [CrossRef] 
[4] Pardoux, E. and Tang, S. (1999) Forward-Backward Stochastic Differential Equations and Quasilinear Parabolic PDEs. Probability Theory and Related Fields, 114, 123-150. [Google Scholar] [CrossRef] 
[5] Yong, J. and Zhou, X.Y. (1999) Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York.
[6] Zhang, J. (2017) Backward Stochastic Differential Equations. Springer, New York, 86.
[7] Han, J.Q. and E, W.N. (2016) Deep Learning Approximation for Stochastic Control Problems.
[8] E, W.N., Han, J.Q. and Jentzen, A. (2017) Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations. Communications in Mathematics and Statistics, 5, 349-380. [Google Scholar] [CrossRef] 
[9] Han, J., Jentzen, A. and E, W.N. (2018) Solving High-Dimensional Partial Differential Equations Using Deep Learning. Proceedings of the National Academy of Sciences, 115, 8505-8510. [Google Scholar] [CrossRef] [PubMed]
[10] Berner, J., Grohs, P. and Jentzen, A. (2020) Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black—Scholes Partial Differential Equations. SIAM Journal on Mathematics of Data Science, 2, 631-657. [Google Scholar] [CrossRef] 
[11] Grohs, P., Jentzen, A. and Salimova, D. (2019) Deep Neural Network Approximations for Monte Carlo Algorithms.
[12] Beck, C., Hornung, F., Hutzenthaler, M., et al. (2020) Overcoming the Curse of Dimensionality in the Numerical Approximation of Allen-Cahn Partial Differential Equations via Truncated Full-History Recursive Multilevel Picard Approximations. Journal of Numerical Mathematics, 28, 197-222. [Google Scholar] [CrossRef] 
[13] Zang, Y., Bao, G., Ye, X., et al. (2020) Weak Adversarial Networks for High-Dimensional Partial Differential Equations. Journal of Computational Physics, 411, Article ID: 109409. [Google Scholar] [CrossRef] 
[14] Fujii, M., Takahashi, A. and Takahashi, M. (2019) Asymptotic Expansion as Prior Knowledge in Deep Learning Method for High Dimensional BSDEs. Asia-Pacific Financial Markets, 26, 391-408. [Google Scholar] [CrossRef] 
[15] Naito, R. and Yamada, T. (2020) An Acceleration Scheme for Deep Learning-Based BSDE Solver Using Weak Expansions. International Journal of Financial Engineering, 7, Article ID: 2050012. [Google Scholar] [CrossRef] 
[16] Øksendal, B. (2003) Stochastic Differential Equations. Springer, Berlin, 21-74.
[17] Goodfellow, I., Bengio, Y. and Courville, A. (2016) Deep Learning. The MIT Press, Cambridge, 151-153, 197-203, 306-310.
[18] Saito, K. 深度学习入门: 基于Python的理论与实现[M]. 陆宇杰, 译. 北京: 人民邮电出版社, 2018: 121-162.
[19] Tang, S. (2015) Dynamic Programming for General Linear Quadratic Optimal Stochastic Control with Random Coefficients. SIAM Journal on Control and Optimization, 53, 1082-1106. [Google Scholar] [CrossRef] 
[20] Tang, S. (2003) General Linear Quadratic Optimal Stochastic Control Problems with Random Coefficients: Linear Stochastic Hamilton Systems and Backward Stochastic Riccati Equations. SIAM Journal on Control and Optimization, 42, 53-75. [Google Scholar] [CrossRef] 
[21] Chang, H. and Rong, X. (2013) An Investment and Consumption Problem with CIR Interest Rate and Stochastic Volatility. Abstract and Applied Analysis, 2013, Article ID: 219397. [Google Scholar] [CrossRef] 
[22] 雷英杰. MATLAB遗传算法工具箱及应用[M]. 西安: 西安电子科技大学出版社, 2005.
[23] Price, K.V., Storn, R., Lampinen, J.A., et al. (2011) Differential Evolution: A Practical Approach to Global Optimization with 48 Tabeles. Springer, Berlin.
[24] Das, S. and Suganthan, P.N. (2011) Differential Evolution: A Survey of the State-of-the-Art. IEEE Transactions on Evolutionary Computation, 15, 4-31. [Google Scholar] [CrossRef] 
[25] Mckinney, W. 利用Python进行数据分析[M/OL]. 徐敬一, 译. 北京: 机械工业出版社, 2018.
https://zh.1lib.tw/book/5422102/5e0803?signAll=1&ts=1826, 2022-01-19.
[26] Idris, I., 张驭宇. Python数据分析基础教程: NumPy学习指南[M]. 第2版. 北京: 人民邮电出版社, 2014.
[27] 李荣华, 刘播. 微分方程数值解法[M]. 北京: 高等教育出版社, 2009: 107-149.
[28] Brigo, D. and Mercurio, F. (2006) Interest Rate Models—Theory and Practice: With Smile, Inflation and Credit. Springer-Verlag, Berlin.

为你推荐