Mining Minimal Failure-Causing Schema for Software Complex Configuration Space

doi:10.23940/ijpe.19.12.p17.32533261

Int J Performability Eng ›› 2019, Vol. 15 ›› Issue (12): 3253-3261.doi: 10.23940/ijpe.19.12.p17.32533261

Previous Articles Next Articles

Mining Minimal Failure-Causing Schema for Software Complex Configuration Space

Liangfen Wei^a, Yong Li^b,c,*, Yong Wang^b,c,d,*, Xiangyu Chen^e, and Zhaohui Xu^f

^aDepartment of Computer Engineering, Anhui Sanlian University, Hefei, 23060, China;
^bKey Laboratory of Safety-Critical Software (Nanjing University of Aeronautics and Astronautics), Ministry of Industry and Information Technology, Nanjing, 210000, China;
^cData Security Key Laboratory, Xinjiang Normal University, Urumqi, 830054, China;
^dState Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210000, China;
^eSchool of Computer and Information, Anhui Polytechnic University, Wuhu, 241000, China;
^fAvic East Optoelectronics (Shanghai) Co., LTD., Shanghai, 201100, China

Submitted on ; Revised on ; Accepted on
Contact: * E-mail address: liyong@live.com, yongwang@ahpu.edu.cn

Abstract

Abstract: Minimal failure-causing schema (MFS) may be affected by masking effects in software complex configuration space. A method for mining MFS based on combination testing and its testing results is proposed. The method firstly constructs a combination-fault tree (CFF-tree) based on the combined test suite and its results, extracts the frequent parameter-value-combinations from the tree as suspicious MFS and calculates their suspiciousness scores, and finally sorts them according to their suspicious scores. Though an iterative framework, MFS is repeatedly mined and checked by programmers until a certain stopping criterion is satisfied. Simulation experiments are used to validate the effectiveness of our method with and without masking effects. The experimental results show that the proposed method can mine MFS in the two scenarios and effectively reduce the number of additional test cases.

Key words: minimal failure-causing schema, configuration space, CFF-tree, combination testing

Liangfen Wei, Yong Li, Yong Wang, Xiangyu Chen, and Zhaohui Xu. Mining Minimal Failure-Causing Schema for Software Complex Configuration Space [J]. Int J Performability Eng, 2019, 15(12): 3253-3261.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

[1] C. Nie and H. Leung, “A Survey of Combinatorial Testing,” ACM Computing Surveys (CSUR), Vol. 43, No. 2, pp. 11, 2011
[2] R. Kuhn, R. Kacker,Y. Lei, “Combinatorial Software Testing,” Computer, Vol. 42, No. 8, pp. 94-96, 2009
[3] L. S. G.Ghandehari, Y. Lei, T. Xie, D. R. Kuhn, and R. Kacker, “Identifying Failure-Inducing Combinations in a Combinatorial Test Set,” inProceedings of IEEE Fifth International Conference on Software Testing, Verification and Validation, pp. 370-379, Montreal, Canada, April 2012
[4] C. Nie and H. Leung, “The Minimal Failure-Causing Schema of Combinatorial Testing,” ACM Transactions on Software Engineering and Methodology (TOSEM), Vol. 20, No. 4, pp. 15, 2011
[5] B. W. Xu, C. Nie, L. Shi,H. -W. Chen, “A Software Failure Debugging Method based on Combinatorial Design Approach for Testing,” Chinese Journal of Computers-Chinese Edition, Vol. 29, No. 1, pp. 132, 2006
[6] Z. Zhang and J. Zhang, “Characterizing Failure-Causing Parameter Interactions by Adaptive Testing,” inProceedings of the 2011 International Symposium on Software Testing and Analysis, pp. 331-341, Toronto, Canada, July 2011
[7] X. Niu, C. Nie, Y. Lei,A. Chan, “Identifying Failure-Inducing Combinations using Tuple Relationship,” inProceedings of 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, pp. 271-280, Luxembourg, Luxembourg, March 2013
[8] C. Yilmaz, S. Fouche, M. B. Cohen, et al., “Moving Forward with Combinatorial Interaction Testing,” Computer, Vol. 47, No. 2, pp. 37-45, 2013
[9] C. Yilmaz, M. B. Cohen,A. A. Porter, “Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces,” IEEE Transactions on Software Engineering, Vol. 32, No. 1, pp. 20-34, 2006
[10] E. Dumlu, C. Yilmaz, M. B. Cohen,A. Porter, “Feedback Driven Adaptive Combinatorial Testing,” inProceedings of the 2011 International Symposium on Software Testing and Analysis, pp. 243-253, Toronto, Canada, July 2011
[11] K. Shakya, T. Xie, N. Li, Y. Lei, R. Kacker,D. R. Kuhn, “Isolating Failure-Inducing Combinations in Combinatorial Testing using Test Augmentation and Classification,” inProceedings of 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, pp. 620-623, Montreal, Canada, April 2012
[12] P. Ammann and J. Offutt, “Introduction to Software Testing,” Cambridge University Press, 2016
[13] J. Han, J. Pei,M. Kamber, “Data Mining: Concepts and Techniques,” Elsevier, 2011
[14] L. S. Ghandehari, J. Chandrasekaran, Y. Lei,L. Sh, “BEN: A Combinatorial Testing-based Fault Localization Tool,” inProceedings of 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1-4, Graz, Austria, April 2015
[15] X. Niu, C. Nie, Y. Lei, K. N. L.Hareton, and X. Wang, “Identifying Failure-Causing Schemas in the Presence of Multiple Faults,” IEEE Transactions on Software Engineering, June 2018

Mining Minimal Failure-Causing Schema for Software Complex Configuration Space

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 1

Recommended 0