基于近端策略优化算法多段压裂水平井CO2吞吐注采优化
CO2 Huff and Puff Rates Optimization of Multi-Stage Fracturing Horizontal Wells Based on Proximal Policy Optimization Algorithm
摘要: CO2吞吐是致密油藏多段压裂水平井衰竭弹性开发后续提高采收率重要接替手段，吞吐注采优化具有成本低、易实现和效果明显的优点。目前吞吐注采优化方法存在不足，未充分考虑不同吞吐轮次和吞闷吐不同阶段间的干扰，本文建立了基于近端策略优化算法多段压裂水平井CO2吞吐注采优化新方法，以净现值为优化目标，吞吐注采参数为优化变量，新方法实现了不同吞吐轮次变注采速度和变注采时长的动态注采优化，充分考虑了吞吐各阶段之间的干扰。实例Y区块CO2吞吐注采优化结果表明：最优方案通过降低返排速度延长返排时间，充分提高返排阶段动用程度，提高了注入CO2吞吐效率，减少吞吐轮次大幅降低注气成本，能够获得最优经济效益，为现场实际注采优化提供指导。
Abstract: CO2 huff and puff is an important replacement method for the subsequent improvement of oil recovery in the elastic development of multi-stage fracturing horizontal wells in tight reservoirs. The rates optimization of huff and puff injection and production has the advantages of low cost, easy implementation, and obvious effects. At present, the rates optimization method of huff and puff injection and production is insufficient, and the interference between different huff and puff cycles and different stages is not fully considered. A novel CO2 huff and puff injection and production rates optimization method for multi-stage fractured horizontal wells based on the proximal policy optimization algorithm has been proposed. With the net present value as the optimization goal and huff and puff injection and production rates parameters as the optimization variables, the new method realizes dynamic injection and production rates optimization with different huff and puff cycles and variable injection and production speed and variable injection and production duration, and considers the interference between various stages. The injection and production rates optimization results of Y block CO2 huff and puff indicate that the optimal project extends the backflow time by reducing the backflow speed, fully improving the utilization degree of the backflow stage. At the same time, improving the efficiency of CO2 injection and reducing the number of cycles can significantly reduce gas injection costs, achieving optimal economic benefits and providing guidance for on-site actual CO2 huff and puff injection and production rates optimization.
文章引用：李荣涛, 曹小朋, 张东, 李宗阳, 王传飞, 韩凤蕊, 郭祥. 基于近端策略优化算法多段压裂水平井CO2吞吐注采优化[J]. 石油天然气学报, 2025, 47(2): 283-293. https://doi.org/10.12677/jogt.2025.472032

参考文献

[1] 张君峰, 毕海滨, 许浩, 等. 国外致密油勘探开发新进展及借鉴意义[J]. 石油学报, 2015, 36(2): 127-137.
[2] 贾承造, 邹才能, 李建忠, 等. 中国致密油评价标准主要类型基本特征及资源前景[J]. 石油学报, 2012, 33(3): 343-350.
[3] 宋俊强, 李晓山, 尤浩宇, 等. 玛湖砾岩油藏复模态结构下的压裂水平井压力分析[J]. 科学技术与工程, 2022, 22(19): 8295-8303.
[4] Ding, M., Gao, M., Wang, Y., Qu, Z. and Chen, X. (2019) Experimental Study on CO2-EOR in Fractured Reservoirs: Influence of Fracture Density, Miscibility and Production Scheme. Journal of Petroleum Science and Engineering, 174, 476-485. [Google Scholar] [CrossRef] 
[5] Zuloaga, P., Yu, W., Miao, J. and Sepehrnoori, K. (2017) Performance Evaluation of CO2 Huff-N-Puff and Continuous CO2 Injection in Tight Oil Reservoirs. Energy, 134, 181-192. [Google Scholar] [CrossRef] 
[6] 侯广. 致密油体积压裂水平井CO2吞吐实践与认识[J]. 大庆石油地质与开发, 2018, 37(3): 163-167.
[7] 何应付, 赵淑霞, 刘学伟. 致密油藏多级压裂水平井CO2吞吐机理[J]. 断块油气田, 2018, 25(6): 752-756.
[8] 杨正明, 刘学伟, 张仲宏, 等. 致密油藏分段压裂水平井注二氧化碳吞吐物理模拟[J]. 石油学报, 2015, 36(6): 724-729.
[9] Sun, J., Zou, A., Sotelo, E. and Schechter, D. (2016) Numerical Simulation of CO2 Huff-N-Puff in Complex Fracture Networks of Unconventional Liquid Reservoirs. Journal of Natural Gas Science and Engineering, 31, 481-492. [Google Scholar] [CrossRef] 
[10] Alfarge, D., Wei, M., Bai, B. and Almansour, A. (2017) Effect of Molecular-Diffusion Mechanisim on CO2 Huff-N-Puff Process in Shale-Oil Reservoirs. SPE Kingdom of Saudi Arabia Annual Technical Symposium and Exhibition, Dammam, 24-27 April 2017, SPE-188003-MS. [Google Scholar] [CrossRef] 
[11] Sanchez-Rivera, D., Mohanty, K. and Balhoff, M. (2015) Reservoir Simulation and Optimization of Huff-and-Puff Operations in the Bakken Shale. Fuel, 147, 82-94. [Google Scholar] [CrossRef] 
[12] Yu, W., Zhang, Y., Varavei, A., Sepehrnoori, K., Zhang, T., Wu, K., et al. (2019) Compositional Simulation of CO2 Huff ’n’ Puff in Eagle Ford Tight Oil Reservoirs with CO2 Molecular Diffusion, Nanopore Confinement, and Complex Natural Fractures. SPE Reservoir Evaluation & Engineering, 22, 492-508. [Google Scholar] [CrossRef] 
[13] Li, L., Zhang, Y. and Sheng, J.J. (2017) Effect of the Injection Pressure on Enhancing Oil Recovery in Shale Cores during the Co2 Huff-N-Puff Process When It Is above and Below the Minimum Miscibility Pressure. Energy & Fuels, 31, 3856-3867. [Google Scholar] [CrossRef] 
[14] Chen, C. and Gu, M. (2017) Investigation of Cyclic CO2 Huff-and-Puff Recovery in Shale Oil Reservoirs Using Reservoir Simulation and Sensitivity Analysis. Fuel, 188, 102-111. [Google Scholar] [CrossRef] 
[15] Sutton, R.S. and Barto, A.G. (2018) Reinforcement Learning: An Introduction. MIT Press.
[16] Peters, J. (2010) Policy Gradient Methods. Scholarpedia, 5, 3698. [Google Scholar] [CrossRef] 
[17] Talavera, A.L., Túpac, Y.J. and Vellasco, M.M. (2010) Controlling Oil Production in Smart Wells by MPC Strategy with Reinforcement Learning. SPE Latin American and Caribbean Petroleum Engineering Conference, Lima, 1-3 December 2010, Peru, SPE-139299-MS. [Google Scholar] [CrossRef] 
[18] Nasir, Y. (2020) Deep Reinforcement Learning for Field Development Optimization. arXiv: 2008.12627.
[19] Nasir, Y., He, J., Hu, C., Tanaka, S., Wang, K. and Wen, X. (2021) Deep Reinforcement Learning for Constrained Field Development Optimization in Subsurface Two-Phase Flow. Frontiers in Applied Mathematics and Statistics, 7, Article 689934. [Google Scholar] [CrossRef] 
[20] Rongtao, L., Liao, X., Wang, X., Zhang, Y., Mu, L., Dong, P., et al. (2022) A Multi-Agent Deep Reinforcement Learning Method for CO2 Flooding Rates Optimization. Energy Exploration & Exploitation, 41, 224-245. [Google Scholar] [CrossRef] 
[21] Schulman, J., Wolski, F,. Dhariwal, P., et al. (2017) Proximal Policy Optimization Algorithms. arXiv: 1707.06347.
[22] Schulman, J., Levine, S., Abbeel, P., et al. (2015) Trust Region Policy Optimization. arXiv: 1502.05477.