Empirical Characterization of the Likelihood of Vulnerability Discovery

doi:10.23940/ijpe.20.07.p3.10081018

Abstract

Abstract: Assessing the risk of the likelihood of a vulnerability discovery is very important for decision-makers to prioritize which vulnerability should be investigated and fixed first. Currently, the likelihood of vulnerability discovery is being assessed based on expert opinion which could potentially hinder its accuracy. In this study, we propose using Time to Vulnerability Disclosure (TTVD) as a proxy for assessing the likelihood of vulnerability discovery. We will then empirically explore characterizing TTVD using intrinsic vulnerability attributes including CVSS Base metrics and vulnerabilities types. We examine 799 reported vulnerabilities of Chrome and 156 vulnerabilities of the Apache HTTP server. The results show that TTVD correlated at a statistically significant level to some of the intrinsic attributes, namely, access complexity metric, confidentiality, and integrity metrics, and the vulnerabilities' types. Our results from machine learning analysis also show ranges of TTVD values are associated with specific combined values of the metrics under consideration.

Key words: likelihood of vulnerability discovery, CWSS, OWASP, software vulnerability, CVSS base score, vulnerability lifecycle, machine learning

Carl Wilhjelm, Taslima Kotadiya, and Awad A. Younis. Empirical Characterization of the Likelihood of Vulnerability Discovery [J]. Int J Performability Eng, 2020, 16(7): 1008-1018.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

1. B. Martin, “Common Weakness Scoring System (CWSS),” The Mitre Corporation, June 2011
2. OWASP Risk Rating Methodology,(https://owasp.org/www-community/OWASP_Risk_Rating_Methodology, accessed May 20 2020)
3. A. Younis, Y. K. Malaiya,I. Ray, “Assessing Vulnerability Exploitability Risk using Software Properties,” Software Quality Journal, Vol. 24, No. 1, pp. 159-202, DOI: 10.1007/s11219-015-9274-6, March 2016
4. M. Bozorgi, L. K. Saul, S. Savage,G. M. Voelker, “Beyond Heuristics: Learning to Classify Vulnerabilities and Predict Exploits,” inProceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 105-113, DOI: 10.1145/1835804.1835821, 2010
5. S. F. Accenture, B. P. E.Zurich, and B. T. E. Zurich, “Modeling the Security Ecosystem-The Dynamics of (In)Security PRIvacy-Aware Secure Monitoring (PRISM) View Project BETEUS View Project,”Springer, pp. 79-106, DOI: 10.1007/978-1-4419-6967-5_6, 2010
6. S. Frei, M. May, U. Fiedler,B. Plattner, “Large-Scale Vulnerability Analysis,” inProceedings of the 2006 SIGCOMM Workshop on Large-scale Attack Defense, LSAD'06, Vol. 2006, pp. 131-138, DOI: 10.1145/1162666.1162671, 2006
7. L. Allodi and F. Massacci, “A Preliminary Analysis of Vulnerability Scores for Attacks in Wild: The EKITS and SYM Datasets,” inProceedings of the ACM Conference on Computer and Communications Security, pp. 17-24, DOI: 10.1145/2382416.2382427, 2012
8. K. Nayak, D. Marino, P. Efstathopoulos, and T. Dumitraş, “Some Vulnerabilities are Different than Others: Studying Vulnerabilities and Attack Surfaces in the Wild,” in Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 8688 LNCS, pp. 426-446, DOI: 10.1007/978-3-319-11379-1_21, 2014
9. C. Sabottke, O. Suciu, T. Dumitraş,T. Dumitras, “Vulnerability Disclosure in the Age of Social Media: Exploiting Twitter for Predicting Real-World Exploits,” inProceedings of the 24th USENIX Conference on Security Symposium, pp. 1041-1056, August 2015
10. A. Younis, Y. K. Malaiya, C. Anderson,I. Ray, “To Fear or Not to Fear that is the Question: Code Characteristics of a Vulnerable Function with an Existing Exploit,” in Proceedings of the 6th ACM Conference on Data and Application Security and Privacy, pp. 97-104, DOI: 10.1145/2857705.2857750, March 2016
11. A. Younis, Y. K. Malaiya, and I. Ray, “Using Attack Surface Entry Points and Reachability Analysis to Assess the Risk of Software Vulnerability Exploitability,” in Proceedings of 2014 IEEE 15th International Symposium on High-Assurance Systems Engineering, HASE 2014, DOI: 10.1109/HASE.2014.10, 2014
12. M. McQueen, T. McQueen, W. Boyer,M. Chaffin, “Empirical Estimates and Observations of 0day Vulnerabilities,” (https://ieeexplore.ieee.org/abstract/document/4755605/?casa_token=gf4z5-32oO0AAAAA:6pl3f2yzMR9fGaYm0ap_lXafVqQ CCO4qNiIWl9qzBhxdaEBk2MyATwANDYDzD_LT0hfea8AshQ, accessed January 2009)
13. NVD - Home, (https://nvd.nist.gov/, accessed May 21 2020)
14. O. S.-S, “Guidelines for Security Vulnerability Reporting and Response. c2004,” (http://www.oisafety. org/guidelines/Guidelines, accessed May 21 2020
15. The CERT Division | Software Engineering Institute,(https://www.sei.cmu.edu/about/divisions/cert/index.cfm, accessed May 21 2020)
16. W. Arbaugh, W. Fithen,J. McHugh, “Windows of Vulnerability: A Case Study Analysis,” (https://ieeexplore.ieee.org /abstract/document/889093/?casa_token=Cp2JuRWLF5EAAAAA:7jNmY5s8n5WgsHYItCvV-vnjoWpaB_eOZxqYY-71gXesT6yn6Gw85MFKS04Lrd59s46PjPWUmg, accessed December 2000)
17. P. Mell, K. Scarfone,S. Romanosky, “A Complete Guide to the Common Vulnerability Scoring System Version 2.0,” (http://www.first.org/cvss/cvss-guide.pdf, accessed June 2007
18. CWE - Common Weakness Enumeration,(https://cwe.mitre.org/, accessed May 21 2020)
19. M. Hafiz and M. Fang, “Game of Detections: How are Security Vulnerabilities Discovered in the Wild?” Empirical Software Engineering, Vol. 21, No. 5, pp. 1920-1959, DOI: 10.1007/s10664-015-9403 7, October 2016
20. Google Chrome Version History - Wikipedia,(https://en.wikipedia.org/wiki/Google_Chrome_version_history, accessed May 21 2020)
21. Chrome Releases, (https://chromereleases.googleblog.com/, accessed May 21 2020)
22. Welcome! - The Apache HTTP Server Project,(https://httpd.apache.org/, accessed May 21 2020)
23. Apache HTTP Server - Wikipedia,(https://en.wikipedia.org/wiki/Apache_HTTP_Server, accessed May 21, 2020)
24. S. -C. -G, Newsletter and undefined 2000, “Full Disclosure and the Window of Exposure,” (https://www.mendeley.com /catalogue/fceceeb1-8021-30a1-aac6-0da5b105200b/, accessed June 2014)
25. S. Muegge and S. Murshed, “Time to Discover and Fix Software Vulnerabilities in Open Source Software Projects: Notes on Measurement and Data Availability,” (https://ieeexplore.ieee.org/abstract/document/8481833/?casa_token=_AOGGP7YAnsAA AAA:Agnz012T8OxA1Dh7YIbuy_PcujWbWvkDst89Wdyo7ha-ftHXn9Y2ebP5Ccr_xRuD9TP-spJmHg, accessed October 2018)
26. H. Joh and Y. K. Malaiya, “Defining and Assessing Quantitative Security Risk Measures Using Vulnerability Lifecycle and CVSS Metrics,” (http://www.cs.colostate.edu/~malaiya/p/johrisk11.pdf, accessed May 22 2020
27. T. Sommestad, H. Holm,M. Ekstedt, “Effort Estimates for Vulnerability Discovery Projects,” (https://ieeexplore.ieee.org/ abstract/document/6149570/?casa_token=ohd5jKeIcGkAAAAA: oNn-H1sJjUJwmTo-Kea6RX47pomKJ-yQt0iZckT3uTnMFC 9Tgin_rYQkXtJsWguIdhNMSZTRug, accessed February 2012)
28. H. Holm, M. Ekstedt,T. Sommestad, “Effort Estimates on Web Application Vulnerability Discovery,” (https://ieeexplore.ieee.org/abstract/document/6480453/?casa_token=7HDCaZgP05gAAAAA:P5pu2w0RwFa77WSSea1hgRu0UkwRE5BZhL9PLtqg0skzoi1sh0midPinDyN16Z2I3wdDQ1IDpA, accessed March 2013)
29. An Empirical Study of Vulnerability Rewards Programs - Google Scholar,(https://scholar.google.com/scholar?hl= en&as_sdt=0%2C11&q=An+empirical+study+of+vulnerability+rewards+programs&btnG=, accessed May 22 2020)
30. A. Younis, Y. K. Malaiya,I. Ray, “Evaluating CVSS base Score using Vulnerability Rewards Programs,”ICT Systems Security and Privacy Protection, Vol. 471, pp. 62-75, 2016
31. YEARFRAC Function - Office Support,(https://support.office.com/en-us/article/yearfrac-function-3844141e-c76d-4143-82b6-208454ddc6a8, accessed May 25 2020)
32. A. M.De Guyon, “An Introduction to Variable and Feature Selection André Elisseeff,” (http://www.jmlr.org/papers/v3/ guyon03a.html, accessed 2003
33. M. Zhao, J. Grossklags,P. Liu, “An Empirical Study of Web Vulnerability Discovery Ecosystems,” inProceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1105-1117, DOI: 10.1145/2810103.2813704, October 2015
34. L. Glanz, S. Schmidt, S. Wollny,B. Hermann, “A Vulnerability's Lifetime: Enhancing Version Information in CVE Databases,” inProceedings of the 15th International Conference on Knowledge Technologies and Data-Driven Business, No. 28, pp. 1-4, DOI: 10.1145/2809563.2809612, 2015

[1]	Ashu Mehta, Navdeep Kaur, and Amandeep Kaur. A Review of Software Fault Prediction Techniques in Class Imbalance Scenarios [J]. Int J Performability Eng, 2025, 21(3): 123-130.
[2]	Vikas, Charu Wahi, Bharat Bhushan Sagar, and Manisha Manjul. Trust Management in WSN using ML for Detection of DDoS Attacks [J]. Int J Performability Eng, 2025, 21(3): 157-167.
[3]	Arpna Saxena and Sangeeta Mittal. CluSHAPify: Synergizing Clustering and SHAP Value Interpretations for Improved Reconnaissance Attack Detection in IIoT Networks [J]. Int J Performability Eng, 2025, 21(1): 36-47.
[4]	Seema Kalonia and Amrita Upadhyay. Comparative Analysis of Machine Learning Model and PSO Optimized CNN-RNN for Software Fault Prediction [J]. Int J Performability Eng, 2025, 21(1): 48-55.
[5]	Vikas Kumar, Charu Wahi, Bharat Bhushan Sagar, and Manisha Manjul. Ensemble Learning Based Intrusion Detection for Wireless Sensor Network Environment [J]. Int J Performability Eng, 2024, 20(9): 541-551.
[6]	Kalyani H. Deshmukh, Gajendra R. Bamnote, and Pratik K Agrawal. A Novel Approach for Drought Monitoring and Evaluation using Time Series Analysis and Deep Learning [J]. Int J Performability Eng, 2024, 20(8): 498-509.
[7]	Saurabh Saxena, and Chetna Gupta. Optimizing Bug Resolution: A Data-Driven Developer Recommendation System [J]. Int J Performability Eng, 2024, 20(8): 510-519.
[8]	Lakshya Vaswani, Sai Sri Harsha, Subham Jaiswal, and Aju D. Unravelling Complexity: Investigating the Effectiveness of SHAP Algorithm for Improving Explainability in Network Intrusion System Across Machine and Deep Learning Models [J]. Int J Performability Eng, 2024, 20(7): 421-431.
[9]	Meenakshi Chawla and Meenakshi Pareek. A Hybrid Deep Learning Perspective for Software Effort Estimation [J]. Int J Performability Eng, 2024, 20(7): 442-450.
[10]	Ajeet Kumar Sharma and Rakesh Kumar. IoT Malware Detection and Dynamic Analysis of MQTT Simulated Network [J]. Int J Performability Eng, 2024, 20(7): 451-459.
[11]	Abhishek Gupta and Jaspreet Singh. Data-Driven Security Framework for VANET using Firefly and ANN [J]. Int J Performability Eng, 2024, 20(6): 344-354.
[12]	Vikas Verma, Arun Malik, and Isha Batra. Analyzing and Classifying Malware Types on Windows Platform using an Ensemble Machine Learning Approach [J]. Int J Performability Eng, 2024, 20(5): 312-318.
[13]	Harshita Batra and Leema Nelson. ESD: E-mail Spam Detection using Cybersecurity-Driven Header Analysis and Machine Learning based Content Analysis [J]. Int J Performability Eng, 2024, 20(4): 205-213.
[14]	Manu Jyoti Gupta and Parveen Sehgal. Optimizing Credit Card Fraud Detection: Classifier Performance and Feature Selection Empowered by Grasshopper Algorithm [J]. Int J Performability Eng, 2024, 20(3): 177-185.
[15]	Aparna Shrivastava and P Raghu Vamsi. Improving Anomaly Classification using Combined Data Transformation and Machine Learning Methods [J]. Int J Performability Eng, 2024, 20(2): 68-80.