The effect of Bellwether analysis on software vulnerability severity prediction models

Kudjo, Patrick Kwaku; Chen, Jinfu; Mensah, Solomon; Amankwah, Richard; Kudjo, Christopher

doi:10.1007/s11219-019-09490-1

Cited by 24 publications

(19 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Classifying Vulnerability Categories: This topic includes publications that construct models to classify the categories of vulnerabilities. For example, P39 [5] classified vulnerabilities based on the predicted severity level of the vulnerabilities using ML models on historical vulnerability data.…”

Section: --P7 [58] Proposed a Novel Approach Called 'Ltrwes'mentioning

confidence: 99%

“…P28 [62] used word embeddings and a onelayer shallow convolutional neural network (CNN) to automatically capture discriminative words and sentence features of bug report descriptions. P39 [5] presented another framework for vulnerability severity classification using the Bellwether analysis (i.e., exemplary data) [63]. They applied the NLP techniques on bug report descriptions.…”

Section: --P7 [58] Proposed a Novel Approach Called 'Ltrwes'mentioning

confidence: 99%

“…We perform an SMS following the guidelines of Petersen et al [7] to synthesize publications that use security bug reports for software vulnerability research. First, we search five scholar databases namely, IEEE Xplore 3 , ACM Digital Library 4 , ScienceDirect 5 , Wiley Online Library 6 , and Springer Link 7 . Using two search strings, we obtain a set of 45,077 publications from the software engineering (SE) domain that were published from 2000 to August 2020.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Security Bug Report Usage for Software Vulnerability Research: A Systematic Mapping Study

Bhuiyan¹,

Sharif²,

Rahman³

2021

IEEE Access

View full text Add to dashboard Cite

Context: Security bug reports are reports from bug tracking systems that include descriptions and resolutions of security vulnerabilities that occur in software projects. Researchers use security bug reports to conduct research related to software vulnerabilities. A mapping study of publications that use security bug reports can inform researchers on (i) the research topics that have been investigated, and (ii) potential research avenues in the field of software vulnerabilities. Objective: The objective of this paper is to help researchers identify research gaps related to software vulnerabilities by conducting a systematic mapping study of research publications that use security bug reports. Method: We perform a systematic mapping study of research that use security bug reports for software vulnerability research by searching five scholar databases: (i) IEEE Xplore, (ii) ACM Digital Library, (iii) ScienceDirect, (iv) Wiley Online Library, and (v) Springer Link. From the five scholar databases, we select 46 publications that use security bug reports by systematically applying inclusion and exclusion criteria. Using qualitative analysis, we identify research topics investigated in our collected set of publications. Results: We identify three research topics that are investigated in our set of 46 publications. The three topics are: (i) vulnerability classification; (ii) vulnerability report summarization; and (iii) vulnerability dataset construction. Of the studied 46 publications, 42 publications focus on vulnerability classification. Conclusion: Findings from our mapping study can be leveraged to identify research opportunities in the domains of software vulnerability classification and automated vulnerability repair techniques.

show abstract

Section: --P7 [58] Proposed a Novel Approach Called 'Ltrwes'mentioning

confidence: 99%

Section: --P7 [58] Proposed a Novel Approach Called 'Ltrwes'mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Security Bug Report Usage for Software Vulnerability Research: A Systematic Mapping Study

Bhuiyan¹,

Sharif²,

Rahman³

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In such a skewed learning environment the testing sample may also contain the same distribution as the training sample. However, many of the existing bug severity prediction models trained from history data are nether mention the distribution of the datasets used to train the models, nor acknowledged this issue [7,8,9,10,11,12,13,14]. This implies that they assumed the underlying data distribution is balanced, which may not essentially be true in all real datasets such as I use in this project.…”

Section: Introductionmentioning

confidence: 98%

“…Hence, the objective of this project is to develop bug severity prediction models from imbalanced learning environments and test them properly using an appropriate metric. Many models developed in the literature [7,8,9,10,11,12,13,14] used the accuracy--correctly classified instances vs. the total number of instances presented--to evaluate the prediction quality in skewed learning environments [4,5]. However, the accuracy would not provide fair judgment on the quality of models trained from unevenly distributed datasets like used in this project.…”

Section: Introductionmentioning

confidence: 99%

Bug Severity Prediction using Keywords in Imbalanced Learning Environment

Ekanayake¹

2021

IJITCS

View full text Add to dashboard Cite

Reported bugs of software systems are classified into different severity levels before fixing them. The number of bug reports may not be equally distributed according to the severity levels of bugs. However, most of the severity prediction models developed in the literature assumed that the underlying data distribution is evenly distributed, which may not correct at all instances and hence, the aim of this study is to develop bug classification models from unevenly distributed datasets and tested them accordingly. To that end first, the topics or keywords of developer descriptions of bug reports are extracted using Rapid Keyword Extraction (RAKE) algorithm and then transferred them into numerical attributes, which combined with severity levels constructs datasets. These datasets are used to build classification models; Naïve Bayes, Logistic Regression, and Decision Tree Learner algorithms. The models’ prediction quality is measured using Area Under Recursive Operative Characteristics Curves (AUC) as the models learnt from more skewed environments. According to the results, the prediction quality of the Logistics Regression model is 0.65 AUC whereas the other two models recorded maximum 0.60 AUC. Though the datasets contain comparatively less number of instances from the high severity classes; Blocking and High, the Logistic Regression models predict the two classes with a decent AUC value of 0.65 AUC. Hence, this projects shows that the models can be trained from highly skewed datasets so that the models prediction quality is equally well over all the classes regardless of number of instances representing the class. Further, this project emphasizes that the models should be evaluated using the appropriate metrics when the models are trained from imbalance learning environments. Also, this work uncovers that the Logistic Regression model is also capable of classifying documents as Naïve Bayes, which is well known for this task.

show abstract

A classification scheme to improve conclusion instability using Bellwether moving windows

Mensah

Kudjo

2022

J Software Evolu Process

Self Cite

View full text Add to dashboard Cite

Context The use of a subset of recently completed and exemplary data, namely, Bellwether moving window (BMW) has proven successful to result in improved accuracy in software effort estimation (SEE). These outcomes were achieved based on the theory that estimation outcome of a future event depends on previous events. Thus, the existence of a BMW yield improved prediction accuracy for new project estimation. However, the conclusion instability problem across learners still threatens the reliability of SEE for new projects. Such instability concerns are attributed to the data subset considered for the training and validation needs of learners. Objective To investigate whether the use of BMWs together with an effort classification scheme can minimize the conclusion instability problem across learners. Method We apply a Bellwether method comprising of three operators, namely, SORT+CLUSTER, GENERATE_TPM, and APPLY to sample the BMW from a pool of chronological projects from the Maxwell and International Software Benchmarking Standards Group (ISBSG) datasets. The sampled BMW is benchmarked against the entire collection of preprocessed projects, namely, growing portfolio to evaluate prediction and classification accuracy across a set of learners–ElasticNet regression, deep neural networks, and automatically transformed linear model. Results (1) BMW exists in the studied projects and (2) training the learners with a BMW of average window size 28.5%–75.5% of the growing portfolio (not older than 3 years) relatively minimizes the conclusion instability of prediction results. Conclusion When BMWs are available, we recommend their use for estimating the effort for a new project to minimize the conclusion instability problem.

show abstract

The effect of Bellwether analysis on software vulnerability severity prediction models

Cited by 24 publications

References 61 publications

Security Bug Report Usage for Software Vulnerability Research: A Systematic Mapping Study

Security Bug Report Usage for Software Vulnerability Research: A Systematic Mapping Study

Bug Severity Prediction using Keywords in Imbalanced Learning Environment

A classification scheme to improve conclusion instability using Bellwether moving windows

Contact Info

Product

Resources

About