Performance Analysis of Software Aging Prediction

doi:10.23940/ijpe.18.11.p15.26922701

Abstract

Abstract: Software aging is a problem that was discovered two decades ago. Since then, many research studies have investigated how to manage aging problems caused by memory leakage and accumulated round-off error through resource consumption prediction or state forecasting. When applying state prediction, the performances of various aging classification algorithms are compared by the prediction error. Since forecasting error is not a precise measure and must be estimated, the forecast error variance needs to be analyzed. In this work, we carefully analyze the forecast error variance by three steps. In the first step, we propose a method to decompose the variance by considering the influence of the data sampling process and data partition procedure. In the second step, we use an enhanced Friedman test and the Nemenyi post hoc test to analyze the influence of the data sampling process on the data partitioning procedure. In the last step, a corrected t-test is proposed to compare the performance of two off-the-shelf classification algorithms. The software comparison experiment is based on a real-time web environment. We end this work by proposing a set of feasible suggestions.

Key words: software aging analysis, variance decomposition, data sampling process, data partition

Yongquan Yan. Performance Analysis of Software Aging Prediction [J]. Int J Performability Eng, 2018, 14(11): 2692-2701.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

[1] J. Zhao, K. S. Trivedi, M. Grottke,Y Wang, “Ensuring the Performance of Apache HTTP Server Affected by Aging,” IEEE Transactions on Dependable and Secure Computing, Vol. 11, No. 2, pp. 130-141, 2014
[2] Z. Zhou, Z. Hu, J. Yu, A. Jemal,C. Morshed, “Energy-Efficient Virtual Machine Consolidation Algorithm in Cloud Data Centers,” Journal of Central South University, Vol. 24, No. 10, pp. 2331-2341, 2017
[3] A. Avritzer and E. J. Weyuker, “Monitoring Smoothly Degrading Systems for Increased Dependability,” Empirical Software Engineering, Vol. 2, No. 1, pp. 59-77, 1997
[4] Y. Huang, C. Kintala, N. Kolettis,N. D. Fulton, “Software Rejuvenation: Analysis, Module and Applications,” inProceedings of IEEE Twenty-Fifth International Symposium on Fault-Tolerant Computing, pp. 381-390, 1995
[5] J. Alonso, R. Matias, E. Vicente, A. Maria,K. S. Trivedi, “A Comparative Experimental Study of Software Rejuvenation Overhead,” Performance Evaluation, Vol. 70, No. 3, pp. 231-250, 2013
[6] I. H. Laradji, M. Alshayeb,L. Ghouti, “Software Defect Prediction Using Ensemble Learning on Selected Features,”Information and Software Technology, Vol. 58, pp. 388-402, 2015
[7] R. S. Wahono, “Integrasi Bagging Dan Greedy Forward Selection Pada Prediksi Cacat Software Dengan Menggunakan Naive Bayes,” Journal of Software Engineering, Vol. 1, No. 2, pp. 101-108, 2015
[8] J. Alonso, L. Belanche,D. R. Avresky, “Predicting Software Bomalies Using Machine Learning Techniques,” inProceedings of IEEE 10th International Symposium on Network Computing and Applications, pp. 163-170, 2011
[9] J. P.Magalhães and L. M. Silva, “Prediction of Performance Anomalies in Web-Applications based-on Software Aging Scenarios,” in Proceedings of IEEE Second International Workshop on Software Aging and Rejuvenation, pp. 1-7, 2010
[10] A. Gulenko, M. Wallschläger, F. Schmidt, O. Kao,F. Liu, “A System Architecture for Real-Time Anomaly Detection in Large-Scale NFV Systems,”Procedia Computer Science, Vol. 94, pp. 491-496, 2016
[11] U. Braga-Neto, “Small-Sample Error Estimation: Mythology Versus Mathematics,” inProceedings of the International Society for Optics and Photonics, pp. 59160V, 2005
[12] A. Zollanvari, U. M.Braga-Neto, and E. R. Dougherty, “On the Sampling Distribution of Resubstitution and Leave-One-Out Error Estimators for Linear Classifiers,” Pattern Recognition, Vol. 42, No. 11, pp. 2705-2723, 2009
[13] J. H. Kim, “Estimating Classification Error Rate: Repeated Cross-Validation, Repeated Hold-Out and Bootstrap,” Computational Statistics & Data Analysis, Vol. 53, No. 11, pp. 3735-3745, 2009
[14] Y. Bengio and Y. Grandvalet, “No Unbiased Estimator of the Variance ofK-Fold Cross-Validation,” Journal of Machine Learning Research, Vol. 5, pp. 1089-1105, 2004
[15] B. Efron, “Bootstrap Methods: Another Look at the Jackknife,” Breakthroughs in Statistics, Springer, New York, NY, pp. 569-593, 1992
[16] R. Kohavi, “A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection,” IJCAI, Vol. 14, No. 2, pp. 1137-1145, 1995
[17] C. Nadeau and Y. Bengio, “Inference for the Generalization Error,” Machine Learning, Vol. 52, No. 3, pp. 239-281, 2003
[18] C. Drummond and R. C. Holte, “C4. 5, Class Imbalance, and Cost Sensitivity: Why Under-Sampling Beats Over-Sampling,” in Proceedings of Workshop on Learning from Imbalanced Datasets II, Washington DC: Citeseer, pp. 1-8, 2003
[19] G. Hepner, T. Logan, N. Ritter,N. Bryant, “Artificial Neural Network Classification Using a Minimal Training Set- Comparison to Conventional Supervised Classification,” Photogrammetric Engineering and Remote Sensing, Vol. 56, No. 4, pp. 469-473, 1990
[20] T. Pohlert, “The Pairwise Multiple Comparison of Mean Ranks Package,”The Comprehensive R Archive Network (CRAN), pp. 2004-2006, 2014
[21] M. Friedman, “A Comparison of Alternative Tests of Significance for the Problem of M Rankings,” The Annals of Mathematical Statistics, Vol. 11, No. 1, pp. 86-92, 1940
[22] R. L.Iman and J. M. Davenport, “Approximations of the Critical Region of the Fbietkan Statistic,” Communications in Statistics-Theory and Methods, Vol. 9, No. 6, pp. 571-595, 1980
[23] J. D. Rodriguez, A. Perez,J. A. Lozano, “Sensitivity Analysis of K-Fold Cross Validation in Prediction Error 7Estimation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, No. 3, pp. 569-575, 2010
[24] Y. Q.Yan and P. Guo, “A Practice Guide of Software Aging Prediction in a Web Server based on Machine Learning,” China Communications, Vol. 13, No. 6, pp. 225-235, 2016