Developing a Modified Fuzzy-GE Algorithm for Enhanced Test Suite Reduction Effectiveness

doi:10.23940/ijpe.23.04.p1.223233

Abstract

Abstract: Software plays a vital role in modern society's application systems. Typically, it is assumed that there are finite faults in the software during software development. However, the development of new functions, such as user requests or other sources, can cause the size of test-case pools to increase, necessitating the reduction of the test suite due to time and resource constraints. To address this issue, this study proposes a Fuzzy Expert System (FES)-based method to improve the fault detection effectiveness of reduced test suites. Specifically, the FES approach is integrated into the traditional test suite reduction technique, the GE algorithm. The performance of our proposed modified Fuzzy-GE algorithm is evaluated based on real programs by using objective criteria. The results show that our approach can effectively reduce the sizes of test suites while also improving the effectiveness of fault detection.

Key words: software testing, test suite reduction, fuzzy expert system, GE algorithm, test case

Chia-En Lai and Chin-Yu Huang. Developing a Modified Fuzzy-GE Algorithm for Enhanced Test Suite Reduction Effectiveness [J]. Int J Performability Eng, 2023, 19(4): 223-233.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

[1] R. S.Pressman and B. R. Maxim, Software engineering: a practitioner's approach, McGraw-Hill, International Edition, 9th edition, 2020.
[2] C. Y. Huang, C. S. Chen,C. E. Lai, Evaluation and analysis of incorporating Fuzzy Expert System approach into test suite reduction, Information and Software Technology, Vol. 79, Issue C, pp. 79-105, November. 2016.
[3] Z. Xu, K. Gao and T. M. Khoshgoftaar, Application of fuzzy expert system in test case selection for system regression test, in Proceedings of IEEE International Conference on Information Reuse and Integration, pp. 120-125, Las Vegas, NV, USA, 2005.
[4] M. J. Harrold, R. Gupta,M. L. Soffa, A methodology for controlling the size of a test suite, ACM Transactions on Software Engineering and Methodology (TOSEM), Vol. 2, No. 3, pp. 270-285, 1993.
[5] J. W.Lin and C. Y. Huang, Analysis of test suite reduction with enhanced tie-breaking techniques, Information and Software Technology, Vol. 51, No. 4, pp. 679-690, April 2009.
[6] C. E. Lai, Improving Fault Detection Capability by Incorporating Fuzzy Expert System Approach into Test Suite Reduction and Test Case Prioritization, MS Thesis, Institute of Information Systems and Applications, National Tsing Hua University, Hsinchu, Taiwan, August. 2016.
[7] A. Kandel, Fuzzy expert systems, Boca Raton, CRC Press, 1991.
[8] A. A. Haider, A. Nadeem,S. Rafiq, Multiple objective test suite optimization: A fuzzy logic based approach, Journal of Intelligent & Fuzzy Systems, Vol. 27, No. 2, pp. 863-875, 2014.
[9] T. Y.Chen and M. F. Lau, A simulation study on some heuristics for test suite reduction, Information and Software Technology, Vol. 40, No. 13, pp. 777-787, 1998.
[10] T. Chen and M. Lau, Heuristics towards the optimization of the size of a test suite,WIT Transactions on Information and Communication Technologies, Vol. 14, 1970.
[11] M. Hutchins, H. Foster, T. Goradia,T. Ostrand, Experiments on the effectiveness of dataflow-and control-flow-based test adequacy criteria, in Proceedings of 16th International conference on Software engineering, Sorrento, Italy, pp. 191-200, May 1994.
[12] F. I.Vokolos and P. G. Frankl, Empirical evaluation of the textual differencing regression testing technique, inProceedings of the International Conference on Software Maintenance (Cat. No. 98CB36272), pp. 44-53, November, 1998.
[13] G. Gharachorlu and N. Sumner, Leveraging Models to Reduce Test Cases in Software Repositories, inProceedings of the 18th International Conference on Mining Software Repositories (MSR), pp. 230-241, Madrid, Spain, May 2021.
[14] E. J.Rapos and J. Dingel, Using Fuzzy Logic and Symbolic Execution to Prioritize UML-RT Test Cases, inProceedings of the 8th International Conference on Software Testing, Verification and Validation (ICST), pp. 1-10, 2015.
[15] M. Harder, J. Mellen,M. D. Ernst, Improving test suites via operational abstraction, in Proceedings of the 25th International Conference on Software Engineering, 2003. Proceedings., pp. 60-71, 2003.
[16] R. Koitz-Hristov, L. Stracke and F. Wotawa, Checked Coverage for Test Suite Reduction - Is It Worth the Effort?, inProceedings of the IEEE/ACM International Conference on Automation of Software Test (AST), pp. 6-16, Pittsburgh, PA, USA, 2022.
[17] J. Becho, F. Cerveira, J. Leitão and R. A. Oliveira, TESRAC: A Framework for Test Suite Reduction Assessment at Scale, inProceedings of IEEE Conference on Software Testing, Verification and Validation (ICST), pp. 174-184, Valencia, Spain, 2022.
[18] C.T. Lin, K. W. Tang,G. M. Kapfhammer, Test suite reduction methods that decrease regression testing costs by identifying irreplaceable tests, Information and Software Technology, Vol. 56, No. 10, pp. 1322-1344, 2014.
[19] O. Jebbar, M. A. Saied, F. Khendek and M. Toeroe, Regression Test Suite Reduction for Cloud Systems, inProceedings of the IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 477-486, Porto, Portugal, 2020.
[20] F. Yan, R. Wu, L. Zhang and Y. Cao, SPIDER: Speeding up Side-Channel Vulnerability Detection via Test Suite Reduction, Tsinghua Science and Technology, vol. 28, no. 1, pp. 47-58, Febrary. 2023.
[21] E. Cruciani, B. Miranda, R. Verdecchia and A. Bertolino, Scalable Approaches for Test Suite Reduction, inProceedings of the 41st International Conference on Software Engineering (ICSE), pp. 419-429, Montreal, QC, Canada, May 2019.
[22] G. Dandan, W. Tiantian, S. Xiaohong,M. Peijun, A test-suite reduction approach to improving fault-localization effectiveness, Computer Languages, Systems & Structures, Vol. 39, No. 3, pp. 95-108, 2013.
[23] P. Kreutzer, T. Kunze and M. Philippsen, Test Case Reduction: A Framework, Benchmark, and Comparative Study, inProceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 58-69, Luxembourg, 2021.
[24] C. Coviello, S. Romano, G. Scanniello, A. Marchetto, G. Antoniol and A. Corazza, Clustering support for inadequate test suite reduction, inProceedings of the 25th International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 95-105, Campobasso, Italy, May 2018.
[25] A. E. V. B. Coutinho, E. G. Cartaxo,P. D. de Lima Machado, Analysis of distance functions for similarity-based test suite reduction in the context of model-based testing, Software Quality Journal, Vol. 24, No. 2, pp. 407-445, 2016.
[26] S. Tallam and N. Gupta, A concept analysis inspired greedy algorithm for test suite minimization, ACM SIGSOFT Software Engineering Notes, Vol. 31, No. 1, pp. 35-42, 2005.
[27] S. Sabharwal, R. Sibal,C. Sharma, Prioritization of test case scenarios derived from activity diagram using genetic algorithm, inProceedings of International Conference on Computer and Communication Technology (ICCCT), pp. 481-485, September, 2010.
[28] D. Kundu, M. Sarma, D. Samanta,R. Mall, System testing for object‐oriented systems with test case prioritization, Software Testing, Verification and Reliability, Vol. 19, No. 4, pp. 297-333, 2009.
[29] N. Chetouane, F. Wotawa, H. Felbinger and M. Nica, On Using k-means Clustering for Test Suite Reduction, inProceedings of IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 380-385, Porto, Portugal, May 2020.
[30] A. Alsharif, G. M.Kapfhammer and P. McMinn, STICCER: Fast and Effective Database Test Suite Reduction Through Merging of Similar Test Cases, inProceedings of the 13th International Conference on Software Testing, Validation and Verification (ICST), pp. 220-230, Porto, Portugal, May 2020.
[31] M. Harrold,G. Rothermel.Aristotle Analysis System-Siemens Programs, HR Variants. Available at: http://www.cc.gatech.edu/aristotle/Tools/subjects
[32] C. L. Chiang, C. Y. Huang, C. Y. Chiu, K. W. Chen,C. H. Lee, Analysis and assessment of weighted combinatorial criterion for test suite reduction,Quality and Reliability Engineering International, 2021. DOI: 10.1002/qre.2984.
[33] X. Zhang, N. Gupta,R. Gupta, Locating faults through automated predicate switching, inProceedings of the 28th International Conference on Software Engineering, pp. 272-281, May 2006.