Automatic Categorization of Software with Document Clustering Methods and Voting Mechanism

doi:10.23940/ijpe.22.04.p3.251262

Abstract

Abstract: Manual software categorization depends on considerable understanding of the categorized software to enable software managers to categorize the software based on certain criteria (e.g., functionality). Unfortunately, the rapid growth of software makes manual software categorization almost impossible and expensive. Therefore, automatic software categorization has become necessary. In this study, we utilized three different unsupervised document clustering methods, namely k-means, non-negative matrix factorization (NMF), and spectral clustering, to analyse source code and to implement automatic software categorization. For evaluation, we selected a well-known unsupervised model, LACT, as our comparison candidate. In general, our proposed methods required at most approximately 1/4 of the execution time of LACT, whereas the fastest method was hundreds of times faster than LACT, achieving at most 26% and 100% better performance based on two criteria: the BCubed F1-measure and the Adjusted Rand Index, respectively. Additionally, we also proposed a voting mechanism inspired by N-version programming and ensemble learning. We selected certain states of NMF and spectral clustering as the referees to improve the average performances of k-means. Our results indicated that combining different clustering techniques to achieve better results is feasible.

Key words: automatic software categorization, document clustering, k-means, nonnegative matrix factorization, spectral clustering

Kai-Wen Chen and Chin-Yu Huang. Automatic Categorization of Software with Document Clustering Methods and Voting Mechanism [J]. Int J Performability Eng, 2022, 18(4): 251-262.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

1. https://github.com, accessed on 2 July 2020.
2. https://sourceforge.net, accessed on 2 July 2020.
3. MLinares-Vásquez, M., McMillan, C., Poshyvanyk, D., and Grechanik, M. On Using Machine Learning to Automatically Classify Software Applications into Domain Categories. Empirical Software Engineering, vol. 19, no. 3, pp. 582-618, 2014.
4. McMillan, C., Linares-Vasquez, M., Poshyvanyk, D., and Grechanik, M. Categorizing Software Applications for Maintenance. In 2011 27th IEEE International Conference on Software Maintenance (ICSM), IEEE, 2011, pp. 343-352.
5. Ugurel S., Krovetz R., andGiles C.L.What's the Code? Automatic Classification of Source Code Archives. InProceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 632-638, 2002.
6. Chen, K.W. and Huang, C.Y.A Study of Applying Unsupervised Learning Methods for Document Clustering and Automatic Categorization of Software. In2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM). IEEE, pp. 1626-1630, 2021.
7. Hartigan, J.A. and Wong, M.A.Algorithm AS 136: A K-means Clustering Algorithm. Journal of the royal statistical society. series c (applied statistics), vol. 28, no. 1, pp. 100-108, 1979.
8. Lee, D. and Seung, H.S.Algorithms for Non-negative Matrix Factorization.Advances in neural information processing systems, 13, 2000.
9. Ng A., Jordan M., andWeiss Y.On Spectral Clustering: Analysis and an Algorithm. Advances in neural information processing systems, vol. 14, 2001.
10. Cui, X., Potok, T.E., and Palathingal, P. Document Clustering using Particle Swarm Optimization. In Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005. IEEE, pp. 185-191, 2005.
11. Sculley D.Web-scale K-means Clustering. InProceedings of the 19th international conference on World wide web, pp. 1177-1178, 2010.
12. Xu W., Liu X., andGong Y.Document Clustering based on Non-negative Matrix Factorization. InProceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pp. 267-273, 2003.
13. Shahnaz F., Berry M.W., Pauca V.P., andPlemmons R.J.Document Clustering using Nonnegative Matrix Factorization. Information Processing & Management, vol. 42, no. 2, pp. 373-386, 2006.
14. Cai D., He X., andHan J.Document Clustering using Locality Preserving Indexing. IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 12, pp. 1624-1637, 2005.
15. Kumar A., Rai P., andDaume H.Co-regularized Multi-view Spectral Clustering. Advances in neural information processing systems, vol. 24, 2011.
16. Bagga, A. and Baldwin, B.Entity-based Cross-document Coreferencing using the Vector Space Model. InCOLING 1998 Volume 1: The 17th International Conference on Computational Linguistics, 1998.
17. Amigó E., Gonzalo J., Artiles J., andVerdejo F.A Comparison of Extrinsic Clustering Evaluation Metrics based on Formal Constraints. Information retrieval, vol. 12, no. 4, pp. 461-486, 2009.
18. Hubert, L. and Arabie, P.Comparing Partitions. Journal of classification, vol. 2, no. 1, pp. 193-218, 1985.
19. O'Grady, K.E. and Medoff, D.R. Categorical Variables in Multiple Regression: Some Cautions. Multivariate behavioral research, vol. 23, no. 2, pp. 243-260, 1988.
20. Campello R.J.A Fuzzy Extension of the Rand Index and Other Related Indexes for Clustering and Classification Assessment. Pattern Recognition Letters, vol. 28, no. 7, pp. 833-841, 2007.
21. Hullermeier, E. and Rifqi, M. A Fuzzy Variant of the Rand Index for Comparing Clustering Structures. In Joint 2009 International Fuzzy Systems Association World Congress and 2009 European Society of Fuzzy Logic and Technology Conference, IFSA-EUSFLAT 2009, pp. 1294-1298, 2009.
22. Avizienis A.The Methodology of N-version Programming. Software fault tolerance, vol. 3, pp. 23-46, 1995.
23. T. Dietterich, Ensemble Learning, The Handbook of Brain Theory and Neural Networks 2, Cambridge, MA, The MIT Press, pp. 110-125, 2002.
24. LeClair, A., Eberhart, Z., and McMillan, C. Adapting Neural Text Classification for Improved Software Categorization. In2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE, pp. 461-472, 2018.
25. Kawaguchi S., Garg P.K., Matsushita M., andInoue K.Mudablue: an Automatic Categorization System for Open Source Repositories. Journal of Systems and Software, vol. 79, no. 7, pp. 939-953, 2006.
26. Tian K., Revelle M., andPoshyvanyk D.Using Latent Dirichlet Allocation for Automatic Categorization of Software. In2009 6th IEEE International Working Conference on Mining Software Repositories, IEEE, pp. 163-166, 2009.
27. Yang, C.Z. and Tu, M.H. Lacta: An Enhanced Automatic Software Categorization on the Native Code of Android Applications. In Proceedings of the international multiconference of engineers and computer scientists (IMECS 2012), vol. 1, 2012.
28. Thomas S.W., Adams B., Hassan A.E., andBlostein D.Studying Software Evolution Using Topic Models.Science of Computer Programming, vol. 80, pp. 457-479, 2014.
29. Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O., Blondel M., Prettenhofer P., Weiss R., Dubourg V., andVanderplas J.Scikit-learn: Machine Learning in Python. the Journal of machine Learning research, vol. 12, pp.2825-2830, 2011.
30. Lloyd S.Least Squares Quantization in PCM. IEEE transactions on information theory, vol. 28, no. 2, pp. 129-137, 1982.
31. Selim, S.Z. and Ismail, M.A.K-means-type algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality. IEEE Transactions on pattern analysis and machine intelligence, vol. 1, pp. 81-87, 1984.
32. Arthur, D. and Vassilvitskii, S., k-means++: The advantages of careful seeding. Stanford, 2006.
33. Berry M.W., Browne M., Langville A.N., Pauca V.P. and Plemmons R.J.Algorithms and Applications for Approximate Nonnegative Matrix Factorization. Computational statistics & data analysis, vol. 52, no. 1, pp. 155-173, 2007.
34. Lee, D. and Seung, H.S.Algorithms for Non-negative Matrix Factorization. Advances in neural information processing systems, vol. 13, 2000.
35. Boutsidis, C. and Gallopoulos, E.SVD Based Initialization: a Head Start for Nonnegative Matrix Factorization. Pattern recognition, vol. 41, no. 4, pp. 1350-1362, 2008.
36. Févotte, C. and Idier, J.Algorithms for Nonnegative Matrix Factorization with the β-divergence. Neural computation, vol. 23, no. 9, pp. 2421-2456, 2011.
37. Stewart, G.W. and Sun, J.G.Matrix Perturbation Theory Academic Press. San Diego, 1990.
38. Von Luxburg, U. A Tutorial on Dpectral Vlustering. Statistics and computing, vol. 17, no. 4, pp. 395-416, 2007.
39. Shi, J. and Malik, J.Normalized Cuts and Image Segmentation. IEEE Transactions on pattern analysis and machine intelligence, vol. 22, no. 8, pp. 888-905, 2000.
40. Bird S., Klein E., andLoper E.Natural language processing with Python: analyzing text with the natural language toolkit. " O'Reilly Media, Inc.", 2009.
41. Neto J.L., Santos A.D., Kaestner C.A., Alexandre N., andSantos D. Document Clustering and Text Summarization, 2000.
42. Hammouda, K.M. and Kamel, M.S.Efficient Phrase-based Document Indexing for Web Document Clustering. IEEE Transactions on knowledge and data engineering, vol. 16, no. 10, pp. 1279-1296, 2004.
43. Han J., Pei J., andKamber M.Data mining: concepts and techniques. Elsevier, 2011.
44. Bollegala D., Matsuo Y., andIshizuka M.Measuring Semantic Similarity Between Words Using Web Search Engines. www, vol. 7, no. 2007, pp. 757-766, 2007.
45. Rosenberg, A. and Hirschberg, J. V-measure: A Conditional Entropy-based External Cluster Evaluation Measure. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL), pp. 410-420, 2007.
46. Rand W.M.Objective Criteria for the Evaluation of Clustering Methods. Journal of the American Statistical association, vol. 66, no. 336, pp. 846-850, 1971.
47. Lutov A., Khayati M., andCudré-Mauroux, P. Accuracy Evaluation of Overlapping and Multi-resolution Clustering Algorithms on Large Datasets. In2019 IEEE International Conference on Big Data and Smart Computing (BigComp), IEEE, pp. 1-8, 2019.

[1]	Deepak Kumar, Chaman Verma, Purushottam Sharma, Deeksha Kumari, and Zoltán Illés. Demographic and Clinical Factors Role Identification in Stroke Risk and Subtype Prediction [J]. Int J Performability Eng, 2023, 19(6): 368-378.
[2]	Ashima Arya and Sanjay Kumar Malik. Software Fault Prediction using K-Mean-Based Machine Learning Approach [J]. Int J Performability Eng, 2023, 19(2): 133-143.
[3]	Arvind Kumar Mishra, Renuka Nagpal, Kirti Seth, and Rajni Sehgal. Maintainability of Service-Oriented Architecture using Hybrid K-means Clustering Approach [J]. Int J Performability Eng, 2023, 19(1): 33-42.
[4]	Kajal Dwivedi, Ramanathan Lakshmanan, and Rajeshkannan Regunathan. K-means Under-Sampling for Hypertension Prediction using NHANES Dataset [J]. Int J Performability Eng, 2021, 17(8): 733-740.
[5]	M. Muzammil Parvez, J. Shanmugam, and V.S. Ghali. Fuzzy C-based Automatic Defect Detection using Barker Coded Thermal Wave Imaging [J]. Int J Performability Eng, 2021, 17(5): 484-490.
[6]	Huaiguang Wu, Pengjie Xie, Ming Cheng, and Hongwei Tao. A Hybrid Model of Predicting Breast Cancer Survivability based on Specific Stages [J]. Int J Performability Eng, 2020, 16(8): 1183-1192.
[7]	Chulei Zhang, Honghua Cui, Yizhang Wang, Tiantian Zhao, and You Zhou. LDKM: An Improved K-Means Algorithm with Linear Fitting Density Peak [J]. Int J Performability Eng, 2020, 16(3): 454-461.
[8]	Lian Yu, Lei Zhang, Cong Tan, Bei Zhao, Chen Zhang, and Lijun Liu. Repeatedly Coding Inter-Packet Delay for Tracking Down Network Attacks [J]. Int J Performability Eng, 2020, 16(2): 265-283.
[9]	Zhiguo Liu and Changqing Ren. Unknown Protocol Data Frame Classification Algorithm based on Improved K-Means [J]. Int J Performability Eng, 2020, 16(11): 1721-1731.
[10]	Qinggang Wu and Xueming Zhai. Remote Sensing Object Detection via an Improved YOLO Network [J]. Int J Performability Eng, 2020, 16(11): 1803-1813.
[11]	Mingzhu Li and Yufeng Deng. A Machine Learning-based Building Operational Pattern Identification [J]. Int J Performability Eng, 2020, 16(11): 1835-1844.
[12]	Wenjie Li. Imbalanced Data Optimization Combining K-Means and SMOTE [J]. Int J Performability Eng, 2019, 15(8): 2173-2181.
[13]	Ming Lei, Bin Wen, Jianhou Gan, and Jun Wang. Clustering Algorithm of Ethnic Cultural Resources based on Spark [J]. Int J Performability Eng, 2019, 15(3): 756-762.
[14]	Yan Zhang, Li Qiao, Xingya Wang, Jingying Cai, and Xuefei Liu. Automatic Software Testing Target Path Selection using K-Means Clustering Algorithm [J]. Int J Performability Eng, 2019, 15(10): 2667-2674.
[15]	Wenqian Shang, Kaixiang Wang, and Junjie Huang. An Improved Tensor Decomposition Model for Recommendation System [J]. Int J Performability Eng, 2018, 14(9): 2116-2126.