基于强化学习的多信道车联网频谱聚合共享

Reinforcement Learning-Based Aggregated Spectrum Sharing for Multi-Channel Vehicular Networking

摘要: 针对车联网需求日益增多以及频谱资源的短缺问题，本文结合认知无线电的频谱聚合功能以及多智能体强化学习方法，提出了基于强化学习的多信道车联网频谱聚合共享模型。每一条车辆到车辆链路作为一个智能体，共同与通信环境交互。各链路独立获得观测结果，同时获得共同的奖励。用这样的设置来促进多个智能体进行合作来训练Q网络，达到改善频谱聚合位置选取和功率分配这一智能体动作的目的。仿真结果表明，通过适当的奖励设计和训练机制，多个智能体能成功学会以分布式方式合作。在不损失车辆到基础设施链路传输总带宽的前提下，本模型能大幅度提高车辆到车辆链路的负载交付率。

Abstract: In response to the increasing demand of vehicular networks and the shortage of spectrum resources, this paper proposes a reinforcement learning-based spectrum aggregation and sharing model for multi-channel vehicular networks by combining the spectrum aggregation function of cognitive radio and a multi-agent reinforcement learning. Each vehicle-to-vehicle link, as an agent, interacts with the communication environment together. Each link obtains observations inde-pendently while receiving a common reward. Such a setup is used to facilitate cooperation among multiple agents to train the Q-network for the purpose of improving spectrum aggregation location picking and power allocation as an agent action. Simulation results show that multiple agents can successfully learn to cooperate in a distributed manner through appropriate reward design and training mechanisms. Without losing the total bandwidth of vehicle-to-infrastructure link transmis-sion, this model can substantially improve the load delivery rate of vehicle-to-vehicle links.