基于强化学习的网络演化博弈合作行为研究

The Study of Cooperative Behaviour in Network Evolutionary Games Based on Reinforcement Learning

摘要: 强化学习因其具有自学习和在线学习能力的特点，日渐成为学者研究演化博弈的重要工具。本文将SARSA算法(State-Action-Reward-State-Action)引入网络博弈中，提出一种基于SARSA算法的演化博弈模型，采用三种强化学习决策机制在四种网络拓扑结构上进行数值仿真模拟。实验表明，引入算法后能明显提高网络中个体的合作水平并且会稳定维持在一个区间范围内。此外，还探讨了算法不同的参数设置、收益矩阵的异质性和个体全局属性对网络合作的影响，结果显示，在学习率较低和折扣率较高以及个体收益适中时对个体间的合作有较好的促进作用。

Abstract: Reinforcement learning is increasingly becoming an important tool for scholars to study evolutionary games due to its features of self-learning and online learning ability. In this paper, the SARSA algorithm (State-Action-Reward-State-Action) is introduced into the network game, and an evolutionary game model based on the SARSA algorithm is proposed, and numerical simulations are conducted on four network topologies using three reinforcement learning decision-making mechanisms. Experiments show that the introduction of the algorithm can significantly improve the level of cooperation of individuals in the network and will be stably maintained in an interval range. In addition, the effects of different parameter settings of the algorithms, the heterogeneity of the payoff matrices and the global attributes of the individuals on the network cooperation are also explored, and the results show that there is a better facilitation of cooperation among individuals at lower learning rates and higher discount rates as well as moderate individual payoffs.