Active Learning using Uncertainty Sampling and Query-by-Committee for Software Defect Prediction

doi:10.23940/ijpe.19.10.p16.27012708

Int J Performability Eng ›› 2019, Vol. 15 ›› Issue (10): 2701-2708.doi: 10.23940/ijpe.19.10.p16.27012708

• Orginal Article • Previous Articles Next Articles

Active Learning using Uncertainty Sampling and Query-by-Committee for Software Defect Prediction

Yubin Qu^a, Xiang Chen^b^*, Ruijie Chen^c, Xiaolin Ju^b, and Jiangfeng Guo^a

^aJiangsu College of Engineering and Technology, Nantong, 226001, China
^bNantong University, Nantong, 226019, China
^cNanjing Foreign Language School, Nanjing, 210008, China

Submitted on ; Revised on ; Accepted on
Contact: Chen Xiang
About author:
* Corresponding author. E-mail address: xchencs@ntu.edu.cn
Supported by:
This work was supported by the Nantong Science and Technology Project (No JC2018134)

Abstract

Abstract:

In the process of software defect prediction dataset construction, there are problems such as high labeling costs. Active learning can reduce labeling costs when using uncertainty sampling. Samples with the most uncertainty will be labeled, but samples with the highest certainty will always be discarded. According to cognitive theory, easy samples can promote the performance of the model. Therefore, a hybrid active learning query strategy is proposed. For the sample with lowest information entropy, query-by-committee will analyze it again using vote entropy. Empirical studies show that the proposed HIVE approach outperforms several state-of-the-art active learning approaches.

Key words: active learning, vote entropy, software defect prediction, uncertainty sampling

Yubin Qu, Xiang Chen, Ruijie Chen, Xiaolin Ju, and Jiangfeng Guo. Active Learning using Uncertainty Sampling and Query-by-Committee for Software Defect Prediction [J]. Int J Performability Eng, 2019, 15(10): 2701-2708.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References 22.

1.	X. Chen, Q. Gu, W. S. Liu, S. L. Liu,C. Ni, “State-of-the-Art Survey of Static Software Defect Prediction,” Ruan Jian Xue Bao/Journal of Software, Vol. 27, No. 1, pp. 1-25, 2016
2.	B. Settles, “Active Learning Literature Survey,” Computer Sciences Technical Report, University of Wisconsin-Madison, pp. 12-25, 2010
3.	S. J. Huang, R. Jin,Z. H. Zhou, “Active Learning by Querying Informative and Representative Examples,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 36, No. 10, pp. 1936-1949, October 2014
4.	C. -L. Li, C. -S. Ferng, and H. -T. Lin, “Active Learning using Hint Information,” Neural Computation, Vol. 27, No. 8, pp. 1738-1765, August 2015
5.	M. Li, H. Zhang,R. Wu, “Sample-based Software Defect Prediction with Active and Semi-Supervised Learning,” Automated Software Engineering, Vol. 19, No. 2, pp. 201-230, June 2012
6.	G. Luo and K. Qin, “Active Learning for Software Defect Prediction,” IEICE Transactions on Information and Systems, Vol. E95-D, No. 6, pp. 680 - 683, June 2012
7.	H. Lu and B. Cukic, “An Adaptive Approach with Active Learning in Software Fault Prediction,” inProceedings of the 8th International Conference on Predictive Models in Software Engineering, pp. 79-88, New York, USA, September 2012
8.	X. Zhou, L. Jin, X. Luo,T. Zhang, “Cross-Version Defect Prediction via Hybrid Active Learning with Kernel Principal Component Analysis,” in Proceedings of2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 209-220, Campobasso, Italy, March 2018
9.	H. Lu, E. Kocaguneli,B. Cukic, “Defect Prediction Between Software Versions with Active Learning and Dimensionality Reduction,” in Proceedings of the2014 IEEE 25th International Symposium on Software Reliability Engineering, pp. 312-322, Washington DC, USA, November 2014
10.	K. Wang, D. Y. Zhang, Y. Li, R. M. Zhang,L. Lin, “Cost-Effective Active Learning for Deep Image Classification,” IEEE Transactions on Circuits and Systems for Video Technology, Vol. 27, No. 12, pp. 2591-2600, December 2017
11.	K. Nigam and A. McCallum, “Employing EM in Pool-based Active Learning for Text Classification,” inProceedings of the Fifteenth International Conference on Machine Learning, pp. 350-358, San Francisco, USA, July 1998
12.	B. Ghotra, S. McIntosh,A. E. Hassan, “Revisiting the Impact of Classification Techniques on the Performance of Defect Prediction Models,” in Proceedings of2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, pp. 789-800, Florence, Italy, May 2015
13.	Z. Xu, J. Liu, Z. J. Yang, G. An,X. Y. Jia, “The Impact of Feature Selection on Defect Prediction Performance: An Empirical Comparison,” in Proceedings of2016 IEEE 27th International Symposium on Software Reliability Engineering, pp. 309-320, Ottawa, Canada, October 2016
14.	W. Yang, X. X. Tian,S. L. WANG, “Recent Advances in Active Learning Algorithms,” Journal of Hebei University (Natural Science Edition), Vol. 37, No. 2, pp. 216-224, 2017
15.	F. Khan, X. Zhu,B. Mutlu, “How Do Humans Teach: On Curriculum Learning and Teaching Dimension,” inProceedings of Twenty-Fifth Conference on Neural Information Processing Systems, pp. 1449-1457, Granada, Spain, December 2011
16.	D. Y. Meng and Q. Zhao, “What Objective Does Self-Paced Learning Indeed Optimize,” arXiv:1511.06049, 2015
17.	M. D'Ambros, M. Lanza,R. Robbes, “An Extensive Comparison of Bug Prediction Approaches,” in Proceedings of2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010), pp. 31-41, Cape Town, South Africa, May 2010
18.	F. Zhang, Q. Zheng, Y. Zou,A. E. Hassan, “Cross-Project Defect Prediction using a Connectivity-based Unsupervised Classifier,” in Proceedings of2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE), pp. 309-320, Austin, USA, May 2016
19.	Y. B.Qu and X. Chen, “ Software Defect Prediction Method based on Cost-Sensitive Active Learning,” Journal of Nantong University (Natural Science Edition), Vol. 18, No. 1, pp. 9-15, February 2019
20.	D. Gray, D. Bowes, N. Davey, Y. Sun,B. Christianson, “The Misuse of the NASA Metrics Data Program Data Sets for Automated Software Defect Prediction,” in Proceedings of 15th Annual Conference on Evaluation and Assessment in Software Engineering (EASE2011), pp. 96-103, Durham, UK, November 2011
21.	T. G. Dietterich, “Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms,” Neural Computation, Vol. 10, No. 7, pp. 1895-1923, October 1998
22.	X. Chen, L. P. Wang, Q. Gu, Z. Wang, C. Ni, W. S. Liu, et al., “A Survey on Cross-Project Software Defect Prediction Methods,” Chinese Journal of Computers, Vol. 41, No. 1, pp. 254-274, January 2018

[1]	K. Eswara Rao, G. Appa Rao, and P. Sankara Rao. A Weighted Ada-Boosting Approach for Software Defect Prediction using Characterized Code Features Associated with Software Quality [J]. Int J Performability Eng, 2022, 18(11): 798-807.
[2]	Mahesha Pandit, Deepali Gupta. Performance of Genetic Programming-based Software Defect Prediction Models [J]. Int J Performability Eng, 2021, 17(9): 787-795.
[3]	Fang Li, Yubin Qu, Junxia Ji, Dejun Zhang, and Long Li. Active Learning Empirical Research on Cross-Version Software Defect Prediction Datasets [J]. Int J Performability Eng, 2020, 16(4): 609-617.
[4]	Yubin Qu, Fang Li, and Xiang Chen. LAL: Meta-Active Learning-based Software Defect Prediction [J]. Int J Performability Eng, 2020, 16(2): 203-213.
[5]	Shibo Wang, Yong Li, Wenbo Mi and Ying Liu. Software Defect Prediction Incremental Model using Ensemble Learning [J]. Int J Performability Eng, 2020, 16(11): 1771-1780.

Active Learning using Uncertainty Sampling and Query-by-Committee for Software Defect Prediction

PDF

Knowledge

Abstract

Cite this article

share this article

References 22.

Related Articles 5

Recommended 0