|
|
|
| Reinforcement learning based energy-efficient internet-of-things video transmission |
Yilin Xiao( ),Guohang Niu(),Liang Xiao*(),Yuzhen Ding(),Sicong Liu(),Yexian Fan() |
Department of Information and Communication Engineering, Xiamen University, Xiamen 361005, China
College of Information and Mechanical and Electrical Engineering, Ningde Normal University, Ningde 352100, China |
|
|
Abstract The video transmission in the Internet-of-Things (IoT) system must guarantee the video quality and reduce the packet loss rate and the delay with limited resources to satisfy the requirement of multimedia services. In this paper, we propose a reinforcement learning based energy-efficient IoT video transmission scheme that protects against interference, in which the base station controls the transmission action of the IoT device including the encoding rate, the modulation and coding scheme, and the transmit power. A reinforcement learning algorithm state-action-reward-state-action is applied to choose the transmission action based on the observed state (the queue length of the buffer, the channel gain, the previous bit error rate, and the previous packet loss rate) without knowledge of the transmission channel model at the transmitter and the receiver. We also propose a deep reinforcement learning based energy-efficient IoT video transmission scheme that uses a deep neural network to approximate value to further accelerate the learning process involved in choosing the optimal transmission action and improve the video transmission performance. Moreover, both the performance bounds of the proposed schemes and the computational complexity are theoretically derived. Simulation results show that the proposed schemes can increase the peak signal-to-noise ratio and decrease the packet loss rate, the delay, and the energy consumption relative to the benchmark scheme.
|
|
Received: 29 September 2020
Online: 19 August 2021
|
| Fund: National Natural Science Foundation of China(61971366);Youth Innovation Fund of Xiamen(3502Z20206039);Natural Science Foundation of Fujian Province of China(2020J01430) |
|
Corresponding Authors:
Liang Xiao
E-mail: ylxiao@stu.xmu.edu.cn;ghniu@stu.xmu.edu.cn;lxiao@xmu.edu.cn;dingding@stu.xmu.edu.cn;liusc@xmu.edu.cn;yfan@ndnu.edu.cn
|
| About author: Yilin Xiao received the BS degree in automation and the MS degree in pattern recognition and intelligent system from Hefei University of Technology, Hefei, China in 2015 and 2018, respectively. He is currently pursuing the PhD degree at the Department of Information and Communication Engineering, Xiamen University, Xiamen, China. His research interests include network security and wireless communications.|Guohang Niu received the BS degree in communication engineering from Xiamen University, Xiamen, China in 2020. He is currently pursuing the MS degree at the Department of Information and Communication Engineering, Xiamen University. His research interests include network security and wireless communications.|Liang Xiao is currently a professor at the Department of Information and Communication Engineering, Xiamen University, Fujian, China. She has served as an associate editor of IEEE Trans. Information Forensics and Security and the guest editor of IEEE Journal of Selected Topics in Signal Processing. She is the recipient of the best paper award for 2016 INFOCOM Big Security WS and 2017 ICC. She received the BS degree in communication engineering from Nanjing University of Posts and Telecommunications, China in 2000, the MS degree in electrical engineering from Tsinghua University, China in 2003, and the PhD degree in electrical engineering from Rutgers University, NJ, USA in 2009. She was a visiting professor at Princeton University, Virginia Tech, and University of Maryland, College Park.|Yuzhen Ding received the BS degree in communication engineering from Wuhan University, Wuhan, China in 2018. She is currently pursuing the MS degree at the Department of Information and Communication Engineering, Xiamen University, Xiamen, China. Her research interests include network security and wireless communications.|Sicong Liu received the BS and PhD degrees with honor both in electronic engineering from Tsinghua University, Beijing, China in 2012 and 2017, respectively. From 2010 to 2011, he was a visiting scholar at City University of Hong Kong. Currently, he is an assistant professor at Department of Information and Communication Engineering, Xiamen University, China. He has published over 50 research papers. He owns 8 Chinese invention patents. He has won the second prize in the Natural Science Award of Chinese Institute of Electronics, and the Best Doctoral Dissertation Award of Tsinghua University. He has served as an associate editor of Frontiers in Communications and Networks, the publication chair of IEEE SmartGridComm 2019, and TPC member of IEEE ICC, Globecom, and several international conferences. His research interests lie in wireless communications, sparse signal processing, machine learning, and visible light communications.|Yexian Fan received the MS degree from Soochow University, Suzhou, China in 2007. Currently, she is an associate professor at the College of Information and Mechanical and Electrical Engineering, Ningde Normal University. Her research interests include wireless network, network security, and machine learning. |
|
|
| [1] |
Ji W., Xu J. C., Qiao H. X., Zhou M. D., and Liang B., Visual IoT: Enabling internet of things visualization in smart cities, IEEE Network, vol. 33, no. 2, pp. 102-110, 2019.
|
| [2] |
Li Z., Liu Y. H., Shin K. G., Liu J., and Yan Z., Interference steering to manage interference in IoT, IEEE Internet of Things Journal, vol. 6, no. 6, pp. 10 458-10 471, 2019.
|
| [3] |
Chen Z. Z. and Pan X., An optimized rate control for low-delay H.265/HEVC, IEEE Trans. Image Process., vol. 28, no. 9, pp. 4541-4552, 2019.
|
| [4] |
Chakareski J., Uplink scheduling of visual sensors: When view popularity matters, IEEE Trans. Commun., vol. 63, no. 2, pp. 510-519, 2015.
|
| [5] |
Wu P. Z., Cosman P. C., and Milstein L. B., Resource allocation for multicarrier device-to-device video transmission: Symbol error rate analysis and algorithm design, IEEE Trans. Commun., vol. 65, no. 10, pp. 4446-4462, 2017.
|
| [6] |
Liu D., Wu J., Cui H., Zhang D. D., Luo C., and Wu F., Cost-distortion optimization and resource control in pseudo-analog visual communications, IEEE Transactions on Multimedia, vol. 20, no. 11, pp. 3097-3110, 2018.
|
| [7] |
Zhou M. L., Wei X. K., Wang S. Q., Kwong S., Fong C. K., Wong P. H. W., and Yuen W. Y. F., Global rate-distortion optimization-based rate control for HEVC HDR coding, IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 12, pp. 4648-4662, 2020.
|
| [8] |
Li B., Li H. Q., Li L., and Zhang J. L., λ domain rate control algorithm for high efficiency video coding, IEEE Trans. Image Process., vol. 23, no. 9, pp. 3841-3854, 2014.
|
| [9] |
Pudlewski S. and Melodia T., A tutorial on encoding and wireless transmission of compressively sampled videos, IEEE Commun. Surv. Tut., vol. 15, no. 2, pp. 754-767, 2013.
|
| [10] |
Xiao L., Zhang H. L., Xiao Y. L., Wan X. Y., Liu S. C., Wang L. C., and Poor H. V., Reinforcement learning-based downlink interference control for ultra-dense small cells, IEEE Transactions on Wireless Communications, vol. 19, no. 1, pp. 423-434, 2020.
|
| [11] |
Sutton R. S. and Barto A. G., Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA, USA: MIT Press, 2018.
|
| [12] |
Wang M. H., Ngan K. N., and Li H. L., Low-delay rate control for consistent quality using distortion-based lagrange multiplier, IEEE Trans. Image Process., vol. 25, no. 7, pp. 2943-2955, 2016.
|
| [13] |
Li C. L., Xiong H. K., and Wu D. P., Delay-rate-distortion optimized rate control for end-to-end video communication over wireless channels, IEEE Trans. Circuits Syst. Video Technol., vol. 25, no. 10, pp. 1665-1681, 2015.
|
| [14] |
Wu J. Y., Cheng B., and Wang M., Adaptive source-FEC coding for energy-efficient surveillance video over wireless networks, IEEE Trans. Commun., vol. 66, no. 5, pp. 2153-2168, 2018.
|
| [15] |
Li P. L., Zhang H. H., Zhao B. H., and Rangarajan S., Scalable video multicast with adaptive modulation and coding in broadband wireless data systems, IEEE/ACM Trans. Netw., vol. 20, no. 1, pp. 57-68, 2012.
|
| [16] |
Yoon J., Zhang H. H., Banerjee S., and Rangarajan S., Video multicast with joint resource allocation and adaptive modulation and coding in 4G networks, IEEE/ACM Trans. Netw., vol. 22, no. 5, pp. 1531-1544, 2014.
|
| [17] |
Gong C. and Wang X. D., Adaptive transmission for delay-constrained wireless video, IEEE Transactions on Wireless Communications, vol. 13, no. 1, pp. 49-61, 2014.
|
| [18] |
Lin S., Miao F., Zhang J. B., Zhou G., Gu L., He T., Stankovic J. A., Son S., and Pappas G. J., ATPC: Adaptive transmission power control for wireless sensor networks, ACM Trans. Sensor Netw., vol. 12, no. 1, p. 6, 2016.
|
| [19] |
Lee N., Lin X. Q., Andrews J. G., and Heath R. W., Power control for D2D underlaid cellular networks: Modeling, algorithms, and analysis, IEEE Journal on Selected Areas in Communications, vol. 33, no. 1, pp. 1-13, 2015.
|
| [20] |
Matthiesen B., Zappone A., Besser K. L., Jorswieck E. A., and Debbah M., A globally optimal energy-efficient power control framework and its efficient implementation in wireless interference networks, IEEE Trans. Signal Process., vol. 68, pp. 3887-3902, 2020.
|
| [21] |
Khalek A. A., Caramanis C., and Heath R. W., Delay-constrained video transmission: Quality-driven resource allocation and scheduling, IEEE Journal of Selected Topics in Signal Processing, vol. 9, no. 1, pp. 60-75, 2015.
|
| [22] |
Mnih V., Kavukcuoglu K., Silver D., Rusu A. A., Veness J., Bellemare M. G., Graves A., Riedmiller M., Fidjeland A. K., Ostrovski G., et al., Human-level control through deep reinforcement learning, Nature, vol. 518, no. 7540, pp. 529-533, 2015.
|
| [23] |
Wang D. W., Toni L., Cosman P. C., and Milstein L. B., Resource allocation and performance analysis for multiuser video transmission over doubly selective channels, IEEE Transactions on Wireless Communications, vol. 14, no. 4, pp. 1954-1966, 2015.
|
| [24] |
Liu Q. W., Zhou S. L., and Giannakis G. B., Cross-layer combining of adaptive modulation and coding with truncated ARQ over wireless links, IEEE Transactions on Wireless Communications, vol. 3, no. 5, pp. 1746-1755, 2004.
|
| [25] |
Jin C., Allen-Zhu Z., Bubeck S., and Jordan M. I., Is Q-learning provably efficient? in Proc. 32nd Int. Conf. Neural Information Processing Systems, Red Hook, NY, USA, 2018, pp. 4868-4878.
|
| [26] |
Ou G. B. and Murphey Y. L., Multi-class pattern classification using neural networks, Pattern Recognition, vol. 40, no. 1, pp. 4-18, 2007.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
