基于改进的深度强化学习策略的交通信号控制

Traffic Signal Control Based on Improved Deep Reinforcement Learning Strategy

摘要: 交叉口的交通信号控制是治理交通拥堵的重要组成部分，而现有的交通信号大多采用循环控制，效率低下且会造成长时间的车辆延迟和能量浪费。针对此问题，采用深度强化学习算法与环境之间进行互动来学习最佳策略。具体地，在智能体学习的初始阶段，创建了一个动作价值评估网络，以增加智能体的学习经验，帮助智能体更快的掌握缓解交通拥堵的技能。提出的模型基于双决斗深度Q网络(Double Dueling Deep Q-Network, 3DQN)算法，车辆的位置信息作为模型的输入，交叉口的四种相位为动作空间，执行动作前后的累积等待时间差被定义为奖励。在城市交通模拟器(Simulation Of Urban Mobility, SUMO)中对模型进行评估。实验结果表明，提出的模型在累积奖励方面相较于DQN、Double DQN、Dueling DQN、3DQN分别增加了58.9%、51.9%、51.3%、48%，证明改进的学习策略可以有效地提升各项交通指标。

Abstract: Traffic signal control at intersections plays a crucial role in managing traffic congestion. However, the conventional cycle control used in existing traffic signals is inefficient and often leads to signifi-cant vehicle delays and energy wastage. To address this issue, a deep reinforcement learning algo-rithm was employed to interact with the environment and learn the optimal control strategy. In the initial stages of the agent’s learning, an action-value evaluation network was established to enhance the agent’s learning experience and facilitate the rapid acquisition of skills for mitigating traffic congestion. The proposed model was based on the double dueling deep Q-Network (3DQN) algo-rithm, utilizing vehicle position information as input and the four phases of the intersection as the action space. The reward was defined as the difference in cumulative waiting time before and after executing an action. The model’s performance was evaluated using the simulation of urban mobility (SUMO) city traffic simulator. Experimental results demonstrated that the proposed model achieves a substantial increase in cumulative rewards, surpassing DQN, double DQN, dueling DQN, and 3DQN by 58.9%, 51.9%, 51.3%, and 48%, respectively. These findings validated the effectiveness of the improved learning strategy in enhancing various traffic indicators.