Regular Articles |
|
|
|
|
Distributed reinforcement learning based framework for energy-effcient UAV relay against jamming |
Weihang Wang,Zefang Lv,Xiaozhen Lu,Yi Zhang*(),Liang Xiao() |
Department of Information and Communication Engineering, Xiamen University, Xiamen 361005, China Beijing Key Laboratory of Mobile Computing and Pervasive Device, Beijing 100190, China |
|
|
Abstract Unmanned aerial vehicle (UAV) network is vulnerable to jamming attacks, which may cause severe damage like communication outages. Due to the energy constraint, the source UAV cannot blindly enlarge the transmit power, along with the complex network topology with high mobility, which makes the destination UAV unable to evade the jammer by flying at will. To maintain communication with a limited battery capacity in the UAV networks in the presence of a greedy jammer, in this paper, we propose a distributed reinforcement learning (RL) based energy-efficient framework for the UAV networks with constrained energy under jamming attacks to improve the communication quality while minimizing the total energy consumption of the network. This framework enables each relay UAV to independently select its transmit power based on historical state-related information without knowing the moving trajectory of other UAVs as well as the jammer. The location and battery level of each UAV need not be shared with other UAVs. We also propose a deep RL based anti-jamming relay approach for UAVs with portable computation equipment like Raspberry Pi to achieve higher and faster performance. We study the Nash equilibrium (NE) and the performance bounds based on the formulated power control game. Simulation results show that the proposed schemes can reduce the bit error rate (BER) and reduce energy consumption of the UAV network compared with the benchmark method.
|
Received: 15 December 2020
Online: 19 August 2021
|
Fund: National Natural Science Foundation of China(61971366);Fundamental Research Funds for the central universities(20720200077) |
Corresponding Authors:
Yi Zhang
E-mail: yizhang@xmu.edu.cn.,;lxiao@xmu.edu.cn
|
About author: Weihang Wang received the BS degree in electronic and information engineering from Hefei University of Technology, China in 2018. She is currently working toward the MS degree at the Department of Information and Communication Engineering, Xiamen University, China. Her research interests include network security, privacy, and wireless communications.|Zefang Lv received the BS degree from Shandong University in 2016 and the MS degree from North China Electric Power University in 2020. She is currently pursuing the PhD degree at the Department of Information and Communication Engineering, Xiamen University, China. Her research interests include network security and wireless communications.|Xiaozhen Lu received the BS degree in communication engineering from Nanjing University of Posts and Telecommunications, Nanjing, China, in 2017. She is currently pursuing the PhD degree at the Department of Information and Communication Engineering, Xiamen University, China. Her research interests include network security and wireless communications.|Yi Zhang received the BS degree in software engineering from Xiamen University in 2014. He received the MS and PhD degrees in communication engineering from Taiwan University in 2016 and 2020, respectively. He was with Quanzhou Institute of Equipment Manufacturing, Chinese Academy of Sciences from 2016 to 2017. He is currently an assistant professor at the Department of Information and Communication Engineering, Xiamen University. His research interests include mobile and wireless networking, fog/edge computing, and game theoretical models for communications networks.|Liang Xiao is currently a professor at the Department of Information and Communication Engineering, Xiamen University, China. She has served as an associate editor of IEEE Trans. Information Forensics and Security and guest editor of IEEE Journal of Selected Topics in Signal Processing. She is the recipient of the best paper award for 2016 INFOCOM Big Security WS and 2017 ICC. She received the BS degree in communication engineering from Nanjing University of Posts and Telecommunications in 2000, the MS degree in electrical engineering from Tsinghua University in 2003, and the PhD degree in electrical engineering from Rutgers University in 2009. She was a visiting professor with Princeton University, Virginia Tech, and University of Maryland, College Park. |
|
|
[1] |
M. Mozaffari, W. Saad, M. Bennis, Y. H. Nam, and M. Debbah, A tutorial on UAVs for wireless networks: Applications, challenges, and open problems, IEEE Commun. Surv. Tuts., vol. 21, no. 3, pp. 2334-2360, 2019.
|
[2] |
Y. Zeng, R. Zhang, and T. J. Lim, Wireless communications with unmanned aerial vehicles: Opportunities and challenges, IEEE Commun. Mag., vol. 54, no. 5, pp. 36-42, 2016.
|
[3] |
L. Gupta, R. Jain, and G. Vaszkun, Survey of important issues in UAV communication networks, IEEE Commun. Surv. Tuts., vol. 18, no. 2, pp. 1123-1152, 2016.
|
[4] |
S. C. Lv, L. Xiao, Q. Hu, X. S. Wang, C. Z. Hu, and L. M. Sun, Anti-jamming power control game in unmanned aerial vehicle networks, in Proc. IEEE Global Communications Conf., Singapore, 2017, pp. 1-6.
|
[5] |
H. Sedjelmaci, S. M. Senouci, and N. Ansari, A hierarchical detection and response system to enhance security against lethal cyber-attacks in UAV networks, IEEE Trans. Syst., Man, Cybern.: Syst., vol. 48, no. 9, pp. 1594-1606, 2018.
|
[6] |
S. D’Oro, E. Ekici, and S. Palazzo, Optimal power allocation and scheduling under jamming attacks, IEEE/ACM Trans. Netw., vol. 25, no. 3, pp. 1310-1323, 2017.
|
[7] |
L. Y. Zhang, Z. Y. Guan, and T. Melodia, United against the enemy: Anti-jamming based on cross-layer cooperation in wireless networks, IEEE Trans. Wirel. Commun., vol. 15, no. 8, pp. 5733-5747, 2016.
|
[8] |
P. Gu, C. Q. Hua, R. Khatoun, Y. Wu, and A. Serhrouchni, Cooperative antijamming relaying for control channel jamming in vehicular networks, IEEE Trans. Veh. Technol., vol. 67, no. 8, pp. 7033-7046, 2018.
|
[9] |
L. Xiao, X. Z. Lu, D. J. Xu, Y. L. Tang, L. Wang, and W. H. Zhuang, UAV relay in VANETs against smart jamming with reinforcement learning, IEEE Trans. Veh. Technol., vol. 67, no. 5, pp. 4087-4097, 2018.
|
[10] |
L. Xiao, D. H. Jiang, D. J. Xu, H. Z. Zhu, Y. Y. Zhang, and H. V. Poor, Two-dimensional antijamming mobile communication based on reinforcement learning, IEEE Trans. Veh. Technol., vol. 67, no. 10, pp. 9499-9512, 2018.
|
[11] |
K. Anazawa, P. Li, T. Miyazaki, and S. Guo, Trajectory and data planning for mobile relay to enable efficient internet access after disasters, in Proc. IEEE Global Communications Conf., San Diego, CA, USA, 2015, pp. 1-6.
|
[12] |
K. Li, W. Ni, X. Wang, R. P. Liu, S. S. Kanhere, and S. Jha, Energy-efficient cooperative relaying for unmanned aerial vehicles, IEEE Trans. Mobile Comput., vol. 15, no. 6, pp. 1377-1386, 2016.
|
[13] |
Y. F. Chen, W. Feng, and G. Zheng, Optimum placement of UAV as relays, IEEE Commun. Lett., vol. 22, no. 2, pp. 248-251, 2018.
|
[14] |
Y. F. Chen, N. Zhao, Z. G. Ding, and M. S. Alouini, Multiple UAVs as relays: Multi-hop single link versus multiple dual-hop links, IEEE Trans. Wirel. Commun., vol. 17, no. 9, pp. 6348-6359, 2018.
|
[15] |
L. Zhang, L. Huang, B. Li, M. Huang, J. W. Yin, and W. M. Bao, Fast-moving jamming suppression for UAV navigation: A minimum dispersion distortionless response beamforming approach, IEEE Trans. Veh. Technol., vol. 68, no. 8, pp. 7815-7827, 2019.
|
[16] |
Z. C. Xiao, B. Gao, S. C. Liu, and L. Xiao, Learning based power control for mmWave massive MIMO against jamming, in Proc. IEEE Global Communications Conf., Abu Dhabi, United Arab Emirates, 2018, pp. 1-6.
|
[17] |
N. Gao, Z. J. Qin, X. J. Jing, Q. Ni, and S. Jin, Anti-intelligent UAV jamming strategy via deep Q-Networks, IEEE Trans. Commun., vol. 68, no. 1, pp. 569-581, 2020.
|
[18] |
L. Xiao, C. X. Xie, M. H. Min, and W. H. Zhuang, User-centric view of unmanned aerial vehicle transmission against smart attacks, IEEE Trans. Veh. Technol., vol. 67, no. 4, pp. 3420-3430, 2018.
|
[19] |
T. S. Rappaport, Wireless Communications: Principles andPractice. Englewood Cliffs, NJ, USA: Prentice-Hall, 1996.
|
[20] |
H. van Hasselt, A. Guez, and D. Silver, Deep reinforcement learning with double Q-learning, in Proc. 30th AAAI Conf. Artificial Intelligence, Phoenix, Arizona, 2016, pp. 2094-2100.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|