Energy-Efficient VM Placement and Replacement through a Hybrid MBFD Q-Learning Framework for Dynamic VM Consolidation

doi:10.23940/ijpe.26.08.p5.473484

Abstract

Abstract: Energy-efficient virtual machine (VM) consolidation is an important part of cloud data center operational overhead reduction. This paper investigates the use of a hybrid consolidation framework based on the Modified Best Fit Decreasing (MBFD) algorithm for initial VM placement and a Q-learning based decision model for dynamic VM reallocation. A combination of the Q-learning and the dual-threshold policy is used to select migration actions based on system states, which include CPU load, power characteristics, and parameters that are related to SLA. A thorough experimental analysis is performed in the MATLAB environment in various workload situations, where the proposed technique is compared to well-established algorithms such as E-ABC, OGrA+OFr and DA-MBFD. The performance trends are analyzed in terms of energy consumption, violations of SLA, number of migrations and utilization of hosts. The results offer a detailed comparison of the behavior of Q-learning-based consolidation as compared to heuristic and metaheuristic strategies, which provides insights into energy-aware resource management in the cloud environment.

Key words: virtual machine (VM), physical machine (PM), cloud computing (CC), modified best fit decreasing (MBFD), service level agreement (SLA), cloud computing data center (CCDC)

Rainu Nandal, Kiran Moar, Rashmi Sindhu, Vipin Kumar, and Kamaldeep Joshi. Energy-Efficient VM Placement and Replacement through a Hybrid MBFD Q-Learning Framework for Dynamic VM Consolidation [J]. Int J Performability Eng, 2026, 22(8): 473-484.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

[1] Srivastava P., andKhan R., 2018. A review paper on cloud computing. International Journal of Advanced Research in Computer Science and Software Engineering,8(6), pp. 17-20.
[2] Buyya R., Yeo C.S., Venugopal S., Broberg J., andBrandic I., 2009. Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Generation Computer Systems,25(6), pp. 599-616.
[3] Shukur H., Zeebaree S., Zebari R., Zeebaree D., Ahmed O., andSalih A., 2020. Cloud computing virtualization of resources allocation for distributed systems. Journal of Applied Science and Technology Trends,1(2), pp. 98-105.
[4] Radha K., Rao B., Babu S., Rao K., Reddy V., andSaikiran P., 2014. Allocation of resources and scheduling in cloud computing with cloud migration. International Journal of Applied Engineering Research,9(19), pp. 5827-5837.
[5] Ghribi C., Hadji M., andZeghlache D., 2013. Energy efficient vm scheduling for cloud data centers: exact allocation and migration algorithms. In2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, pp. 671-678.
[6] Mann Z.Á.,2015. Allocation of virtual machines in cloud data centers—a survey of problem models and optimization algorithms. ACM Computing Surveys (CSUR),48(1), pp. 1-34.
[7] Kamran and Nazir, B., 2018. QoS-aware VM placement and migration for hybrid cloud infrastructure. the Journal of Supercomputing,74(9), pp. 4623-4646.
[8] Barlaskar E., Singh Y.J., andIssac B., 2018. Enhanced cuckoo search algorithm for virtual machine placement in cloud data centres. International Journal of Grid and Utility Computing,9(1), pp. 1-17.
[9] Kaur A., Kumar S., Gupta D., Hamid Y., Hamdi M., Ksibi A., Elmannai H., andSaini S., 2023. Algorithmic approach to virtual machine migration in cloud computing with updated SESA algorithm.Sensors, 23(13), 6117.
[10] Liu N., Li Z., Xu J., Xu Z., Lin S., Qiu Q., Tang J., andWang Y., 2017. A hierarchical framework of cloud resource allocation and power management using deep reinforcement learning. In2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 372-382.
[11] Ghasemi A., Toroghi Haghighat A., andKeshavarzi A., 2023. Enhanced multi-objective virtual machine replacement in cloud data centers: combinations of fuzzy logic with reinforcement learning and biogeography-based optimization algorithms. Cluster Computing,26(6), pp. 3855-3868.
[12] Ibrahim A., Noshy M., Ali H.A., andBadawy M., 2020. PAPSO: A power-aware VM placement technique based on particle swarm optimization.IEEE Access, 8, pp. 81747-81764.
[13] Zeng J., Ding D., Kang K., Xie H., andYin Q., 2022. Adaptive DRL-based virtual machine consolidation in energy-efficient cloud data center. IEEE Transactions on Parallel and Distributed Systems,33(11), pp. 2991-3002.
[14] Muthusamy A., andDhanaraj R.K., 2023. Dynamic Q‐learning‐based optimized load balancing technique in cloud.Mobile Information Systems, 2023(1), 7250267.
[15] Sindhu R., Siwach V., Sehrawat H., andMor K., 2024. Optimizing cloud resource utilization with ANN-based VM placement and prediction.SN Computer Science, 5(7), 907.
[16] Reinecke, P. and Di Marco, A. eds., 2017. Computer Performance Engineering: 14th European Workshop, EPEW 2017, Berlin, Germany, September 7-8, 2017, Proceedings. Springer.
[17] Zhu X., Xia R., Zhou H., Zhou S., andLiu H., 2024. An intelligent decision system for virtual machine migration based on specific Q-learning.Journal of Cloud Computing, 13(1), 122.
[18] Ghasemi A., andToroghi Haghighat A., 2020. A multi-objective load balancing algorithm for virtual machine placement in cloud data centers based on machine learning: A. Ghasemi, AT Haghighat. Computing,102(9), pp. 2049-2072.
[19] Talwani S., Singla J., Mathur G., Malik N., Jhanjhi N.Z., Masud M., andAljahdali S., 2022. Machine-learning-based approach for virtual machine allocation and migration.Electronics, 11(19), 3249.
[20] Sui X., Liu D., Li L., Wang H., andYang H., 2019. Virtual machine scheduling strategy based on machine learning algorithms for load balancing.EURASIP Journal on Wireless Communications and Networking, 2019(1), 160.
[21] Hidayat T., Ramli K., Thereza N., Daulay A., Rushendra R., andMahardiko R., 2024. Machine learning to estimate workload and balance resources with live migration and VM placement. InInformatics, 11(3), 50.
[22] Alourani A., Khalid A., Tahir M., andSardaraz M., 2024. Energy efficient virtual machines placement in cloud datacenters using genetic algorithm and adaptive thresholds.Plos One, 19(1), e0296399.
[23] Long S., Li Z., Xing Y., Tian S., Li D., andYu R., 2020. A reinforcement learning-based virtual machine placement strategy in cloud data centers. In2020 IEEE 22nd International Conference on High Performance Computing and Communications; IEEE 18th International Conference on Smart City; IEEE 6th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), pp. 223-230.
[24] Tong Z., Chen H., Deng X., Li K., andLi K., 2020. A scheduling scheme in the cloud computing environment using deep Q-learning.Information Sciences, 512, pp. 1170-1191.
[25] Hummaida A.R., Paton N.W., andSakellariou R., 2022. Scalable virtual machine migration using reinforcement learning.Journal of Grid Computing, 20(2), 15.
[26] Tran C.H., Bui T.K., andPham T.V., 2022. Virtual machine migration policy for multi-tier application in cloud computing based on Q-learning algorithm. Computing,104(6), pp. 1285-1306.
[27] Beloglazov A., Abawajy J., andBuyya R., 2012. Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Future Generation Computer Systems,28(5), pp. 755-768.
[28] Wu Q., andZhang R., 2018. 2018 IEEE global communications conference (GLOBECOM).