一种基于增强学习的飞行自组网地理路由协议
A Geographic Routing Protocol for FANETs Based on Reinforcement Learning
DOI: 10.12677/CSA.2022.122030, PDF,    科研立项经费支持
作者: 杨 斌：成都信息工程大学计算机学院，四川 成都；王辛果：中国航空工业无线电电子研究所，上海
关键词: FANETs；Q-Learning；地理路由协议；ns3-Gym；FANETs； Q-Learning； Geographic Routing Protocol； ns3-Gym
摘要: 飞行自组网(FANETs，Flying Ad-Hoc Networks的缩写)是航空平台组建的自组网网络，不依赖固定通信设施，具备部署快，健壮性高等优势，可广泛应用于应急通信和军事等场景。然而，由于航空平台的移动速率更高，飞行自组网的网络拓扑动态性更高，现有的移动自组网路由协议无法直接适用。在现有的FANETs的路由协议中，基于地理位置的路由协议相较于其他路由协议具有很大的优势，它仅依靠于节点的地理坐标，不建立和维护端到端连接。但是，现有的基于地理位置的路由协议也存在最优转发节点选择困难和端到端延迟较高等问题。为此，本文提出了一种基于增强学习的地理路由协议，称为QEgr。该协议基于Q-Learning算法综合考虑了链路稳定性和延迟，并使用了ns3-gym模拟器与其他路由协议进行了比较。实验表明，与GPSR、Q-Grid、Q-Geo等经典地理路由协议相比，QEgr具有更低的端到端延迟和更高的数据包发送成功率。
Abstract: FANETs (the abbreviation of Flying Ad-Hoc Networks) is an Ad-Hoc network formed by aviation platforms. It does not rely on fixed communication facilities. It has the advantages of fast deployment and high robustness. It can be widely used in emergency communications and military scenarios. However, due to the higher mobile speed of the aviation platform and the higher dynamics of the network topology of the FANETs, the existing mobile Ad-Hoc network routing protocol cannot be directly applied. Among the existing FANETs routing protocols, the geographical location-based routing protocol has great advantages over other routing protocols. It only relies on the geographical coordinates of the nodes and does not establish and maintain end-to-end connections. However, the existing geographic location-based routing protocols also have problems such as difficulty in selecting the optimal forwarding node and high end-to-end delay. To this end, this paper proposes a geographic routing protocol based on enhanced learning called QEgr. This protocol is based on the Q-Learning algorithm and considers the link stability and delay, and uses the ns3-gym simulator to compare with other routing protocols. Experiments show that, compared with classic geographic routing protocols such as GPSR, Q-Grid, and Q-Geo, QEgr has a lower end-to-end delay and a higher packet transmission success rate.
文章引用：杨斌, 王辛果. 一种基于增强学习的飞行自组网地理路由协议[J]. 计算机科学与应用, 2022, 12(2): 304-314. https://doi.org/10.12677/CSA.2022.122030

参考文献

[1] Wang, J.J., Jiang, C.X., Han, Z., Ren, Y., Maunder, R.G. and Hanzo, L. (2017) Taking Drones to the Next Level: Coop-erative Distributed Unmanned-Aerial-Vehicular Networks for Small and Mini Drones. IEEE Vehicular Technology Magazine, 12, 73-82. [Google Scholar] [CrossRef] 
[2] Alshbatat, A.I. and Dong, L. (2010) Cross Layer Design for Mobile Ad-Hoc Unmanned Aerial Vehicle Communication Networks. 2010 International Conference on Networking, Sensing and Control (ICNSC), Chicago, 10-12 April 2010, 331-336. [Google Scholar] [CrossRef] 
[3] Watkins, C.J. and Dayan, P. (1992) Q-Learning. Machine Learning, 8, 279-292. [Google Scholar] [CrossRef] 
[4] Li, R.L., Li, F., Li, X. and Wang, Y. (2014) Qgrid: Q-Learning Based Routing Protocol for Vehicular Ad Hoc Networks. 2014 IEEE 33rd International Perfor-mance Computing and Communications Conference (IPCCC), Austin, 5-7 December 2014, 1-8. [Google Scholar] [CrossRef] 
[5] Jung, W.-S., Yim, J. and Ko, Y.-B. (2017) Qgeo: Q-Learning-Based Geographic Ad Hoc Routing Protocol for Unmanned Robotic Networks. IEEE Communications Let-ters, 21, 2258-2261. [Google Scholar] [CrossRef] 
[6] Lin, L., Sun, Q., Wang, S. and Yang, F. (2012) A Geo-graphic Mobility Prediction Routing Protocol for Ad Hoc UAV Network. 2012 IEEE Globecom Workshops, Anaheim, 3-7 December 2012 1597-1602. [Google Scholar] [CrossRef] 
[7] Choi, S.-C., Hussen, H.R., Park, J.-H. and Kim, J. (2018) Geolocation-Based Routing Protocol for Flying Ad Hoc Networks (FANETs). 2018 Tenth International Conference on Ubiquitous and Future Networks (ICUFN), Prague, 3-6 July 2018, 50-52. [Google Scholar] [CrossRef] 
[8] Arafat, M.Y. and Moh, S. (2018) Location-Aided Delay Tol-erant Routing Protocol in UAV Networks for Post-Disaster Operation. IEEE Access, 6, 59891-59906. [Google Scholar] [CrossRef] 
[9] He, C.T., Wang, Q., Xu, Y.J., Liu, J.M. and Xu, Y.D. (2019) A Q-Learning Based Cross-Layer Transmission Protocol for MANETs. 2019 IEEE International Conferences on Ubiquitous Computing & Communications (IUCC) and Data Science and Computational Intelligence (DSCI) and Smart Computing, Networking and Services (SmartCNS), Shenyang, 21-23 October 2019, 580-585. [Google Scholar] [CrossRef] 
[10] Lyu, N.Q., Song, G.H., Yang, B.W. and Cheng, Y.N. (2018) Qngpsr: A Q-Network Enhanced Geographic Ad-Hoc Routing Protocol Based on GPSR. 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall), Chicago, 27-30 August 2018, 1-6. [Google Scholar] [CrossRef] 
[11] Sutton, R.S. (1988) Learning to Predict by the Methods of Temporal Differences. Machine Learning, 3, 9-44. [Google Scholar] [CrossRef] 
[12] Gawlowicz, P. & Zubow, A. (2019). ns-3 Meets OpenAI Gym: The Playground for Machine Learning in Networking Research. Proceedings of the 22nd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, Miami Beach, 25-29 November 2019, 113-120.[CrossRef]