Comparing Minimal Failure-Causing Schema and Probabilistic Failure-Causing Schema on Boolean Specifications

doi:10.23940/ijpe.19.10.p17.27092717

Int J Performability Eng ›› 2019, Vol. 15 ›› Issue (10): 2709-2717.doi: 10.23940/ijpe.19.10.p17.27092717

• Orginal Article • Previous Articles Next Articles

Comparing Minimal Failure-Causing Schema and Probabilistic Failure-Causing Schema on Boolean Specifications

Ziyuan Wang^a^b^*, Xueyao Li^a, Yang Li^a, and Yuqing Dai^c

^aSchool of Computer Science and Technology, Nanjing University of Posts and Telecommunications, Nanjing, 210023, China
^bTongda College, Nanjing University of Posts and Telecommunications, Yangzhou, 225127, China
^c School of Information Technology, Nanjing University of Chinese Medicine, Nanjing, 210023, China

Submitted on ; Revised on ; Accepted on
Contact: Wang Ziyuan
About author:
* Corresponding author. E-mail address: wangziyuan@njupt.edu.cn
Supported by:
This work is supported by the National Nature Science Foundation of China (No 61772259) and the Foundation of Nanjing University of Posts and Telecommunications (No NY219079)

Abstract

Abstract:

Both the model of minimal failure-causing schema (MFS) and the model of probabilistic failure-causing schema (PFS) were proposed to describe characteristics of failure test cases in input-domain testing. To improve the efficiency of software debugging, input variables that are related to failure-causing schemas should be closer to the real fault-relevant input variables. In order to examine which model (MFS or PFS) can help software engineers localize fault-relevant input variables more preciously, we conduct an experiment on general-form Boolean specifications extracted from the well-known TCAS system. For each mutant of a general-form Boolean expression, the set of input variables localized by the MFSs, the set of input variables localized by the PFSs, and the set of actual input variables involved in the fault are compared. Experimental results suggest that the MFS model usually has an advantage in terms of recall, while the PFS model usually has an advantage in terms of precision. Overall, the latter has a slight advantage in terms of f-measure.

Key words: input-domain testing, combinatorial testing, minimal failure-causing schema, probabilistic failure-causing schema, fault localization

Ziyuan Wang, Xueyao Li, Yang Li, and Yuqing Dai. Comparing Minimal Failure-Causing Schema and Probabilistic Failure-Causing Schema on Boolean Specifications [J]. Int J Performability Eng, 2019, 15(10): 2709-2717.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References 18.

1.	P. Bourque and R. E. Fairley, “SWEBOK: Guide to the Software Engineering Body of Knowledge (v3.0),” IEEE Computer Society, 2014
2.	A. Zeller and R. Hildebrandt, “Simplifying and Isolating Failure-Inducing Input,” IEEE Transaction on Software Engineering, Vol. 28, No. 2, pp. 183-200, 2002
3.	C. Nie and H. Leung, “The Minimal Failure-Causing Schema of Combinatorial Testing,” ACM Transactions on Software Engineering and Methodology, Vol. 20, No. 4, pp. 15, 2011
4.	Z. Wang, Y. Qi,J. Lin, “Probabilistic Failure-Causing Schema in Input-Domain Testing,” in Proceedings of 27th International Conference on Software Engineering and Knowledge Engineering (SEKE2015), pp. 726, 2015
5.	Z. Chen, T. Y. Chen,B. Xu, “A Revisit of Fault Class Hierarchies in General Boolean Specifications,” ACM Transactions on Software Engineering and Methodology, Vol. 20, No. 3, pp. 13, 2011
6.	D. L.Olson and D. Delen, “Advanced Data Mining Techniques,” Springer Berlin Heidelberg, 2008
7.	N. G. Leveson, M. P. E.Heimdahl, H. Hildreth, and J. D. Reese, “Requirements Specification for Process-Control Systems,” IEEE Transaction Software Engineering, Vol. 20, No. 9, pp. 684-707, 1994
8.	E. Weyuker, T. Goradia,A. Singh, “Automatically Generating Test Data from a Boolean Specification,” IEEE Transaction Software Engineering, Vol. 20, No. 5, pp. 353-363, 1994
9.	N. Kobayashi, T. Tsuchiya,T. Kikuno, “Application of Non-Specification-based Approaches to Logic Testing for Software,” in Proceedings of International Conference on Dependable Systems and Networks (DSN2001), pp. 337-346, 2001
10.	N. Kobayashi, T. Tsuchiya,T. Kikuno, “Non-Specification-based Approaches to Logic Testing for Software,” Information and Software Technology, Vol. 44, No. 2, pp. 113-121, 2002
11.	Z. Wang and Y. Qi, “Why Combinatorial Testing Works: Analyzing Minimal Failure-Causing Schemas in Logic Expressions,” in Proceedings of IEEE 8th International Conference on Software Testing, Verification, and Validation Workshops (ICSTW2015):4th International Workshop on Combinatorial Testing (IWCT2015), April 13-17, 2015
12.	Z. Wang and M. Yu, “BoolMuTest: A Prototype Tool for Fault-based Boolean-Specification Testing,” in Proceedings of 30th International Conference on Software Engineering and Knowledge Engineering (SEKE2018), pp. 720-721, 2018
13.	C. J.Colbourn and D. W. McClary, “Locating and Detecting Arrays for Interaction Faults,” Journal of Combinatorial Optimization, Vol. 15, No. 1, pp. 17-48, 2008
14.	C. Martinez, L. Moura,D. Panario, “Locating Errors Using ELAs, Covering Arrays, and Adaptive Testing Algorithms,” Siam Journal on Discrete Mathematics, Vol. 23, No. 4, pp. 1776-1799, 2010
15.	T. Konishi, H. Kojima,H. Nakagawa, “Finding Minimum Locating Arrays using a SAT Solver,” in Proceedings of IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW2017), 2017
16.	Z. Wang, B. Xu,L. Chen, “Adaptive Interaction Fault Location based on Combinatorial Testing,” inProceedings of International Conference on Quality Software (QSIC2010), pp. 495-502, 2010
17.	Z. Zhang and J. Zhang, “Characterizing Failure-Causing Parameter Interactions by Adaptive Testing,” in Proceedings of the2011 International Symposium on Software Testing and Analysis (ISSTA2011), pp. 331-341, 2011
18.	X. Niu, C. Nie,Y. Lei, “Identifying Failure-Inducing Combinations using Tuple Relationship,” in Proceedings of IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops (ICSTW2013), pp. 271-280, 2013

[1]	Yihao Li, Pan Liu, W. Eric Wong, Nicholas Chau, and Chih-Wei Hsu. Alternative Ranking Distance Metrics for Fault-Focused Clustering in Parallel Fault Localization [J]. Int J Performability Eng, 2023, 19(10): 633-643.
[2]	Ziyuan Wang, Yanliang Zhang, Peng Gao, and Shiyong Shuang. Comparing Fault Detection Efficiencies of Adaptive Random Testing and Greedy Combinatorial Testing for Boolean-Specifications [J]. Int J Performability Eng, 2021, 17(1): 114-122.
[3]	Mengyu Ji, Song Huang, and Zhanwei Hui. Spectrum-based Security Bug Localization by Analyzing Error Propagation [J]. Int J Performability Eng, 2020, 16(8): 1289-1298.
[4]	Yong Wang, SanMing Liu, Jun Li, Xiangyu Cheng, and Wan Zhou. Lightweight Fault Localization using Weighted Dynamic Control Flow Subgraph [J]. Int J Performability Eng, 2020, 16(2): 214-222.
[5]	Zhao Li, Yi Song, Siwei Zhou, Dongcheng Li, and Peng Chen. Normalization of Notation NCF for Improving Fault Localization [J]. Int J Performability Eng, 2019, 15(7): 1988-1997.
[6]	Bingwu Fang, Yong Li, Yong Wang, Xiangyu Cheng, and Zhaohui Xu. Search-based Software Debugging using Weighted Fault Propagation Graphs [J]. Int J Performability Eng, 2019, 15(12): 3179-3186.
[7]	Liangfen Wei, Yong Li, Yong Wang, Xiangyu Chen, and Zhaohui Xu. Mining Minimal Failure-Causing Schema for Software Complex Configuration Space [J]. Int J Performability Eng, 2019, 15(12): 3253-3261.
[8]	Ziyuan Wang, Yang Li, Xueqing Gu, Xiaojia Zheng, and Min Yu. Fault Detection Capabilities of Combinatorial Testing and Random Testing for Boolean-Specifications [J]. Int J Performability Eng, 2019, 15(11): 2952-2961.
[9]	Chia-Hao Lee, Chin-Yu Huang, and Tzu-Yang Lin. A Study of Applying Fault-based Genetic-Like Programming Approaches to Automatic Software Fault Corrections [J]. Int J Performability Eng, 2018, 14(9): 2090-2104.
[10]	Yong Wang, Qiansong Wang, Guifu Lu, Zhiqiu Huang, Bingwu Fang, Yong Li, and Weiwei Li. From Predicate Testing to Identify Fault Location for Safety-Critical Software [J]. Int J Performability Eng, 2018, 14(9): 2066-2075.

Comparing Minimal Failure-Causing Schema and Probabilistic Failure-Causing Schema on Boolean Specifications

PDF

Knowledge

Abstract

Cite this article

share this article

References 18.

Related Articles 10

Recommended 0