D2PG: Deep Deterministic Policy Gradient-Based Vehicular Edge Caching Scheme for Digital Twin-Based Vehicular Networks

doi:10.23940/ijpe.23.05.p7.350358

Int J Performability Eng ›› 2023, Vol. 19 ›› Issue (5): 350-358.doi: 10.23940/ijpe.23.05.p7.350358

D²PG: Deep Deterministic Policy Gradient-Based Vehicular Edge Caching Scheme for Digital Twin-Based Vehicular Networks

Harshvardhan Singh Chauhan, Himanshi Babbar^*, and Shalli Rani

Chitkara University Institute of Engineering and Technology, Chitkara University, Punjab, India

Contact: * E-mail address: himanshi.babbar@chitkara.edu.in
About author:Harshvardhan Singh is currently pursuing B.E degree with the Institute of Technology, Chitkara University, Rajpura, India. His research interests include Software-defined Networking, Digital twin, Edge computing and Metaverse.
Himanshi Babbar is Assistant Professor- Research in CSE, working in Chitkara University, Rajpura, Punjab, India. She has 2 years of teaching experience, CGC, Landran, Mohali, Punjab. She received an MCA (Master's in Computer Applications) degree from Chitkara University, Punjab Campus in 2015 and completed her Ph.D. and Postdoctoral Fellowship in Computer Applications from Chitkara University, Punjab Campus and UAE in 2021 and 2022 respectively. Her research areas are Software Defined Networking, Load Balancing, Deep Learning, Intrusion Detection and Internet of Things. She has served as a reviewer in many conferences and journals including the International Conference on Intelligent and Innovative Technologies in Computing, Electrical and Electronics (ICIITCEE 2023), IEEE International Conference on Current Regards Development in Engineering and Technology (CCET-2022), Peer-to-Peer Networking and Applications, Ad Hoc Networks, Elsevier, Springer's Journal of Network and Systems Management, etc. She has published/accepted/presented many papers at national and international conferences, published over 25 papers in SCI-indexed journals and filed/published/granted more than 15 patents. She received the Young Researcher Award from the Institute of Scholars (INSC) in April 2023.
Shalli Rani is Associate Professor in CSE with Chitkara University (Rajpura (Punjab)), India. She has 14+ years of teaching experience. She received an MCA degree from Maharishi Dayanand University, Rohtak in 2004 and the M.Tech. degree in Computer Science from Janardan Rai Nagar Vidyapeeth University, Udaipur in 2007 and a Ph.D. degree in Computer Applications from Punjab Technical University, Jalandhar in 2017. Her main area of interest and research is Wireless Sensor Networks, Underwater Sensor networks and Internet of Things. She has published/accepted/presented more than 25 papers in international journals /conferences. She has worked on Big Data, Underwater Acoustic Sensors and IoT to show the importance of WSN in IoT applications. She received a young scientist award in Feb. 2014 from Punjab Science Congress, in the same field.

Abstract

Abstract: Digital twin technology has gained significant attention in recent years as a promising approach for improving the performance and efficiency of various systems including vehicular networks. Vehicular networks are critical for intelligent transportation systems, providing communication and coordination among vehicles, roadside infrastructure, and other entities to enable efficient traffic management, enhanced safety, and improved driving experiences. SDN is a potential networking architecture that isolates the control plane from the data plane, enabling centralized administration and control of network resources. SDN provides programmability, flexibility, and scalability, making it well-suited for managing the complex and dynamic nature of vehicular networks. Combining digital twin technology with SDN can enable intelligent management and control of vehicular networks, leading to improved performance, enhanced reliability, and efficient resource utilization. In this paper, a novel framework is proposed that leverages digital twin technology in vehicular networks using SDN. The architecture presented integrates digital twin models with SDN controllers, enabling real-time monitoring, analysis, and control of vehicular network components. The digital twin models are used to represent virtual replicas of physical vehicular components, such as vehicles, roadside units, and traffic signals, providing a holistic view of the vehicular network's behavior and performance. The digital twin models are discussed and can be used for various vehicular network management tasks, including traffic flow optimization, congestion detection and mitigation, predictive maintenance, and incident management.

Key words: software defined networking, digital twin, vehicular networks, intelligent digital twin - software defined vehicular network (IDT SDVN)

Harshvardhan Singh Chauhan, Himanshi Babbar, and Shalli Rani. D²PG: Deep Deterministic Policy Gradient-Based Vehicular Edge Caching Scheme for Digital Twin-Based Vehicular Networks [J]. Int J Performability Eng, 2023, 19(5): 350-358.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

1. Eason G., Noble B., & Sneddon, I. N. On certain integrals of Lipschitz-Hankel type involving products of Bessel functions. Phil. Trans. Roy. Soc. London, A247, pp. 529-551, 1955
2. Clerk Maxwell.A Treatise on Electricity and Magnetism (3rd ed., Vol. 2). Oxford: Clarendon, pp. 68-73, 1892
3. Jacobs I.S.,1963. Fine particles, thin films and exchange anisotropy. Magnetism, pp.271-350.
4. Qiao G., Leng S., Maharjan S., Zhang Y., & Ansari, N. Deep Reinforcement Learning for Cooperative Content Caching in Vehicular Edge Computing and Networks. IEEE Internet of Things Journal, vol. 7, no. 1, 2020
5. Coutinho, R.W. and Boukerche, A.Modeling and analysis of a shared edge caching system for connected cars and industrial IoT-based applications. IEEE Transactions on Industrial Informatics, vol. 16, no. 3, pp. 2003-2012, 2019
6. Ning Z., Zhang K., Wang X., Obaidat M.S., Guo L., Hu X., Hu B., Guo Y., Sadoun B. and Kwok R.Y.Joint computing and caching in 5G-envisioned Internet of vehicles: A deep reinforcement learning-based traffic control system. IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 8, pp.5201-5212, 2020
7. Mahmood A., Casetti C.E., Chiasserini C.F., Giaccone P. and Härri J.The RICH prefetching in edge caches for in-order delivery to connected cars. IEEE Transactions on Vehicular Technology, vol. 68, no. 1, pp. 4-18, 2018
8. Dai Y., Xu D., Zhang K., Maharjan S. and Zhang Y.Deep reinforcement learning and permissioned blockchain for content caching in vehicular edge computing and networks. IEEE Transactions on Vehicular Technology, vol. 69, no. 4, pp.4312-4324, 2020
9. Ning Z., Dong P., Wang X., Hu X., Liu J., Guo L., Hu B., Kwok R.Y. and Leung V.C.Partial computation offloading and adaptive task scheduling for 5G-enabled vehicular networks. IEEE Transactions on Mobile Computing, vol. 21, no. 4, pp.1319-1333, 2020
10. He, Y., Zhao, N. and Yin, H.Integrated networking, caching, and computing for connected vehicles: A deep reinforcement learning approach. IEEE Transactions on Vehicular Technology, vol. 67, no. 1, pp.44-55, 2017
11. Dai Y., Xu D., Maharjan S., Qiao G. and Zhang Y.Artificial intelligence empowered edge computing and caching for internet of vehicles. IEEE Wireless Communications, vol. 26, no. 3, pp.12-18, 2019
12. Ning Z., Zhang K., Wang X., Guo L., Hu X., Huang J., Hu B. and Kwok R.Y.Intelligent edge computing in internet of vehicles: a joint computation offloading and caching solution. IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 4, pp.2212-2225, 2020
13. Yu Z., Hu J., Min G., Zhao Z., Miao W. and Hossain M.S.Mobility-aware proactive edge caching for connected vehicles using federated learning. IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 8, pp.5341-5351, 2020
14. Zhou, J., Zhang, X. and Wang, W.Social-aware proactive content caching and sharing in multi-access edge networks. IEEE Transactions on Cognitive Communications and Networking, vol. 6, no. 4, pp.1308-1319, 2020
15. Radenkovic, M. and Huynh, V.S.H. Cognitive caching at the edges for mobile social community networks: A multi-agent deep reinforcement learning approach.IEEE Access, vol. 8, pp.179561-179574, 2020
16. Prasher S., Nelson L., Sindhu A.S., Sumathi S. and Jagdish M. MCSVM and MCRVM based Contingency Classification Model. In2023 Second International Conference on Electronics and Renewable Systems (ICEARS) (pp. 440-443). IEEE, 2023, March
17. Prasher, S., Nelson, L. and Sharma, A. Analysis of DenseNet201 with SGD optimizer for diagnosis of multiple rice leaf diseases. In2022 International Conference on Computational Modelling, Simulation and Optimization (ICCMSO) (pp. 182-186). IEEE, 2022, December.
18. Rani S., Babbar H., Srivastava G., Gadekallu T.R. and Dhiman G.Security Framework for Internet of Things based Software Defined Networks using Blockchain. IEEE Internet of Things Journal, vol. 10, no. 7, pp. 6074-6081, 2022
19. Sharma A., Chauhan H.S., Kaur H. and Babbar H. Analysis of DDoS Attacks in Software Defined Networking using Machine Learning. In2023 International Conference on Intelligent and Innovative Technologies in Computing, Electrical and Electronics (IITCEE) (pp. 1190-1195). IEEE, 2023, January.

[1]	Khushi Wadhwa and Himanshi Babbar. Digital Twin in the Motorized (Automotive / Vehicle) Industry [J]. Int J Performability Eng, 2023, 19(9): 568-578.
[2]	Neelam Gupta, Sarvesh Tanwar, Sumit Badotra, and Sunny Behal. Performance Analysis of SDN Controller [J]. Int J Performability Eng, 2022, 18(8): 537-544.

D²PG: Deep Deterministic Policy Gradient-Based Vehicular Edge Caching Scheme for Digital Twin-Based Vehicular Networks

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 2

Recommended 0