基于柔性行动器–评判器的电–热综合能源系统协调优化
Coordinated Optimization of Integrated Electricity-Heat Energy System Based on Soft Actor-Critic
摘要: 电–热综合能源系统的优化调度对于实现系统的能源互补、经济运行具有重要意义。本文提出一种基于柔性行动器–评判器(Soft Actor-Critic, SAC)算法的电–热综合能源系统经济调度方法，首先针对电–热综合能源系统优化调度问题进行建模，然后基于SAC框架将该问题转化为强化学习模型，搭建了强化学习环境。最后对基于SAC的电–热综合能源系统优化运行解结果进行分析，并进一步验证该方法的有效性。
Abstract: The optimal dispatch of integrated electricity-heat energy system (IEHS) is of great significance to the energy complementation and economic operation of the system. An economic dispatch method for IEHS based on the Soft Actor-Critic (SAC) algorithm is proposed in this paper. Firstly, an optimal dispatch model of IEHS is established, and then the problem is transformed into a reinforcement learning model based on the SAC framework, and a reinforcement learning environment is built. Finally, the optimal operation result of integrated electricity-heat energy system based on SAC is analyzed, and the simulations show that the proposed method can effectively solve the problem and reduce the operation cost.
文章引用：刘雨, 董雷, 王春斐, 李梦婷, 乔骥, 王新迎. 基于柔性行动器–评判器的电–热综合能源系统协调优化[J]. 智能电网, 2021, 11(2): 107-117. https://doi.org/10.12677/SG.2021.112011

参考文献

[1] Wikipedia (2013) Quantum Entanglement. https://en.wikipedia.org/wiki/Quantum_entanglement
[2] 余晓丹, 徐宪东, 陈硕翼, 吴建中, 贾宏杰. 综合能源系统与能源互联网简述[J]. 电工技术学报, 2016, 31(1): 1-13.
[3] 吴晨雨. 电热综合能源系统的建模及优化运行[D]: [博士学位论文]. 南京: 东南大学, 2019.
[4] Lu, S., Gu, W., Zhou, J.H., Zhang, X.S. and Wu, C.Y. (2018) Coordinated Dispatch of Multi-Energy System with District Heating Network: Modeling and Solution Strategy. Energy, 152, 358-370.
[Google Scholar] [CrossRef] 
[5] Li, Z., et al. (2017) Combined Heat and Power Dispatch Considering Pipeline Energy Storage of District Heating Network. IEEE Transactions on Sustainable Energy, 7, 12-22.
[Google Scholar] [CrossRef] 
[6] Zeng, Y.J. and Sun, Y.G. (2014) An Improved Particle Swarm Optimization for the Combined Heat and Power Dynamic Economic Dispatch Problem. Electric Power Components and Systems, 42, 1700-1716.
[Google Scholar] [CrossRef] 
[7] Yang, J., Zhang, N., Botterud, A. and Kang, C. (2020) On An Equivalent Representation of the Dynamics in District Heating Networks for Combined Electricity-Heat Operation. IEEE Transactions on Power Systems, 35, 560-570.
[Google Scholar] [CrossRef] 
[8] Zhang, B., Huang, Q., Chen, Z. and Blaabjerg, F. (2019) Deep Reinforcement Learning-Based Approach for Optimizing Energy Conversion in Integrated Electrical and Heating System with Renewable Energy. Energy Conversion and Management, 202, Article ID: 112199.
[Google Scholar] [CrossRef] 
[9] Zhang, B., Hu, W.H., Li, J.H., Cao, D., Huang, R., Huang, Q., Chen, Z. and Blaabjerg, F. (2020) Dynamic Energy Conversion and Management Strategy for an Integrated Electricity and Natural Gas System with Renewable Energy: Deep Reinforcement Learning Approach. Energy Conversion and Management, 220, Article ID: 113063.
[Google Scholar] [CrossRef] 
[10] 李宏仲, 王磊, 林冬, 张雪莹. 多主体参与可再生能源消纳的Nash博弈模型及其迁移强化学习求解[J]. 中国电机工程学报, 2019, 39(14): 4135-4150.
[11] Ji, Y., Wang, J.H., Xu, J.C., Fang, X.K. and Zhang, H.G. (2019) Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning. Energies, 12, 2291.
[Google Scholar] [CrossRef] 
[12] Hua, H.C., Chao, Y., Hao, C.T. and Cao, J.W. (2019) Optimal Energy Management Strategies for Energy Internet via Deep Reinforcement Learning Approach. Applied Energy, 239, 598-609.
[Google Scholar] [CrossRef] 
[13] Liu, X.Z., Wu, J.Z., Jenkins, N. and Bagdanavicius, A. (2016) Combined Analysis of Electricity and Heat Networks. Applied Energy, 162, 1238-1250.
[Google Scholar] [CrossRef] 
[14] Haarnoja, T., Zhou, A., Abbeel, P. and Levine, S. (2018) Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. 2018 International Conference on Machine Learning (ICML), Stockholmsmässan, 10-15 July 2018, 1861-1870.
[15] Huang, J., Li, Z. and Wu, Q.H. (2017) Coordinated Dispatch of Electric Power and District Heating Networks: A Decentralized Solution Using Optimality Condition Decomposition. Applied Energy, 206, 1508-1522.
[Google Scholar] [CrossRef] 

为你推荐