基于驾驶风格识别与协作学习的交通信号强化学习方法

A Reinforcement Learning Approach for Traffic Signal Control Integrating Driving Style Recognition and Collaborative Learning

摘要: 随着城市化进程的加快，城市交通拥堵问题日益严峻，已成为制约城市可持续发展的核心瓶颈。近年来，智能交通信号控制技术迅速发展，尤其是基于多智能体深度强化学习(Multi-Agent Deep Reinforcement Learning, MADRL)的方法，为缓解交通拥堵提供了新思路。然而，现有研究普遍忽视了驾驶员行为的异质性，并在大规模交叉口网络中面临状态空间维度过高与智能体协作效率不足的双重挑战。针对上述问题，本文提出一种融合驾驶风格识别与邻居协作机制的多智能体深度强化学习算法——CDS-DQN (Collaborative Driving Style-aware Deep Q-Network)。该算法设计了一种轻量化驾驶风格识别模块，通过量化车辆行为特征(如激进型、普通型、保守型)，构建“有效占有率”指标作为状态输入，以增强对微观交通特性的感知能力。同时，提出了一种邻居状态共享机制，使各智能体能够获取相邻交叉口的关键信息，实现局部协同感知并缓解多智能体系统中的环境非平稳性问题。基于SUMO (Simulation of Urban Mobility)仿真平台构建的城市路网环境中对该算法进行了实验验证。结果表明，CDS-DQN在平均等待时间、队列长度与通行效率等指标上，均优于传统固定配时控制、独立DQN与主流的MA2C算法，充分展示了其有效性与先进性。

Abstract: With the accelerating process of urbanization, urban traffic congestion has become increasingly severe, emerging as a critical bottleneck for sustainable urban development. In recent years, intelligent traffic signal control technology has undergone rapid development, particularly methods based on Multi-Agent Deep Reinforcement Learning (MADRL), which offer new avenues for alleviating traffic congestion. However, existing research often overlooks the heterogeneity of driver behavior and faces the dual challenges of high-dimensional state spaces and inefficient agent collaboration in large-scale intersection networks. To address these issues, this paper introduces a novel Multi-Agent Deep Reinforcement Learning algorithm, the Collaborative Driving Style-aware Deep Q-Network (CDS-DQN). The proposed algorithm features a lightweight driving style recognition module that quantifies vehicle behavioral characteristics (e.g., aggressive, normal, and conservative) to formulate an “effective occupancy” metric. This metric serves as a state input, enhancing the agent’s perceptual capabilities regarding micro-level traffic dynamics. Furthermore, a neighbor state-sharing mechanism is proposed, enabling each agent to access critical information from adjacent intersections. This facilitates local collaborative perception and mitigates the environmental non-stationarity problem inherent in multi-agent systems. The algorithm was experimentally validated in an urban road network environment constructed on the SUMO (Simulation of Urban Mobility) platform. The results demonstrate that CDS-DQN outperforms traditional fixed-time control, independent DQN, and the state-of-the-art MA2C algorithm across key performance indicators, including average waiting time, queue length, and traffic throughput. These findings fully showcase the effectiveness and advanced nature of the proposed algorithm.