Adaptive Job-Scheduling Algorithm based on Queuing Theory in a Hybrid Cloud Environment

doi:10.23940/ijpe.19.06.p9.15801590

Int J Performability Eng ›› 2019, Vol. 15 ›› Issue (6): 1580-1590.doi: 10.23940/ijpe.19.06.p9.15801590

Previous Articles Next Articles

Adaptive Job-Scheduling Algorithm based on Queuing Theory in a Hybrid Cloud Environment

Yanpei Liu^a,*, Xiaoni Chen^a, Ying Hu^a, and Qiang Cai^b

a School of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou, 450002, China
b Beijing Key Laboratory of Big Data Technology for Food Safety, Beijing Technology and Business University, Beijing, 102488, China

Submitted on ;
Contact: * E-mail address: liuyanpei@zzuli.edu.cn
About author:Yanpei Liu is currently a teacher in the School of Computer and Communication Engineering at Zhengzhou University of Light Industry. She received her Ph.D. in computer science and technology from Wuhan University of Technology. She received her M.S. degree from Nanchang Hangkong University in 2009 and her B.S. degree from Luoyang Normal University in 2006. Her research interests include cloud computing and distributed computing;Xiaoni Chen is currently a postgraduate student in the School of Computer and Communication Engineering at Zhengzhou University of Light Industry. She received her B.S. degree from Zhengzhou University of Light Industry in 2017. Her research interests include cloud computing and data mining;Ying Hu is currently a teacher in the School of Computer and Communication Engineering at Zhengzhou University of Light Industry. She received her Ph.D. from Zhengzhou University in 2016. Her research interests include computer networks and distributed computing;Qiang Cai is currently a professor in the School of Computer and Information Engineering at Beijing Technology and Business University. His research interests are web databases, distributed computing, and heterogeneous systems integration.
Supported by:
The authors thank the editor and the anonymous reviewers for their helpful comments and suggestions. The work was supported by the National Natural Science Foundation (No. 61802353), Henan Provincial Department of Science and Technology (No. 192102210270), Open Issues of Beijing City Key Laboratory (No. BKBD-2017KF08), and Dr. Fund of Zhengzhou University of Light Industry.

Abstract

Abstract: To resolve the problem of unreasonable resource allocations caused by the continuous arrival of different types of jobs in a hybrid cloud environment, an adaptive job-scheduling algorithm based on queuing theory is proposed. This paper analyses job load types, and the jobs are classified according to the logistic regression method. A resource utility is used to classify the nodes in a private cloud cluster by considering the heterogeneity of the private cloud resources. Based on the job classification and the resource classification, a queuing model is established, and an adaptive genetic algorithm is used to manage the job queue's arrival rate that becomes the basis of the resource allocation. The proposed algorithm is compared with some existed similar algorithms to verify its performance in terms of job response times and throughput.

Key words: hybrid cloud, heterogeneous resources, queuing theory, job-scheduling

Yanpei Liu, Xiaoni Chen, Ying Hu, and Qiang Cai. Adaptive Job-Scheduling Algorithm based on Queuing Theory in a Hybrid Cloud Environment [J]. Int J Performability Eng, 2019, 15(6): 1580-1590.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

[1] J. G. Chen, K. L. Li, Z. Tang,K. Bilal, “A Parallel Random Forest Algorithm for Big Data in a Spark Cloud Computing Environment,” IEEE Transactions on Parallel & Distributed Systems, Vol. 28, No. 4, pp. 919-933, 2017
[2] C. Napoli, G. Pappalardo,E. Tramontana, “A Cloud-Distributed GPU Architecture for Pattern Identification in Segmented Detectors Big-Data Surveys,” Computer Journal, Vol. 59, No. 3, pp. 338-352, 2018
[3] C. W. Yang, Q. Y. Huang,Z. L. Li, “Big Data and Cloud Computing: Innovation Opportunities and Challenges,” International Journal of Digital Earth, Vol. 10, No. 1, pp. 13-53, 2017
[4] Y. H.Moon and C. H. Youn, “Multihybrid Job Scheduling for Fault-Tolerant Distributed Computing in Policy-Constrained Resource Networks,” Computer Networks, Vol. 82, No. 3, pp. 81-95, 2015
[5] A. Rasooli and D. G. Down, “COSHH: A Classification and Optimization based Scheduler for Heterogeneous Hadoop Systems,” Future Generation Computer Systems, Vol. 36, No. 3, pp. 1-15, 2014
[6] W. Chongdarakul, P. Sophatsathit,C. Lursinsap, “Efficient Task Scheduling based on Theoretical Scheduling Pattern Constrained on Single I/O Port Collision Avoidance,” Simulation Modelling Practice and Theory, Vol. 56, No. 67, pp. 171-190, 2016
[7] M. Khan, Y. Liu, and M. Li, “Data Locality in Hadoop Cluster Systems,” in Proceedings of 2014 International Conference on Fuzzy Systems and Knowledge Discovery, pp. 720-724, 2014
[8] W. Tian, G. Luo,L. Tian, “On Dynamic Job Ordering and Slot Configurations for Minimizing the Makespan of Multiple MapReduce Jobs,” IEEE Transactions on Service Computing, Vol. 9, No. 1, pp. 1-6, 2016
[9] C. T. Chen, L. J. Hung,S. Y. Hsieh, “Heterogeneous Job Allocation Scheduler for Hadoop MapReduce using Dynamic Grouping Integrated Neighboring Search,”IEEE Transactions on Cloud Computing, pp. 1-14, 2017
[10] O. Komori, S. Eguchi,S. Ikeda, “An Asymmetric Logistic Regression Model for Ecological Data,” Methods in Ecology & Evolution, Vol. 7, No. 2, pp. 249-260, 2016
[11] S. Spicuglia and L. Y. Chen, “On Load Balancing: A Mix-Aware Algorithm for Heterogeneous Systems,” in Proceedings of 2013 International Conference on PERFORMANCE Engineering, pp. 71-76, 2013

Adaptive Job-Scheduling Algorithm based on Queuing Theory in a Hybrid Cloud Environment

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 0

Recommended 0