Bug severity prediction using question-and-answer pairs from Stack Overflow

Tan, Youshuai; Xu, Sijie; Wang, Zhaowei; Zhang, Tao; Zhou, Xu; Wang, Liu

doi:10.1016/j.jss.2020.110567

Cited by 48 publications

(30 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Y. Tan et al [37] proposed a novel approach for predicting severity. They linked the bug repositories post on stack overflows to the contents of Mozilla, Eclipse, and GCC bug reports.…”

Section: Volume XX 2020mentioning

confidence: 99%

CaPBug-A Framework for Automatic Bug Categorization and Prioritization Using NLP and Machine Learning Algorithms

2021

View full text Add to dashboard Cite

Bug reports facilitate software development teams in improving the quality of software. These reports include significant information related to problems encountered within a software, possible enhancement suggestions, and other potential issues. Bug reports are typically complex and are too detailed; hence a lot of resources are required to analyze and process them manually. Moreover, it leads to delays in the resolution of high priority bugs. Accurate and timely processing of bug reports based on their category and priority plays a significant role in improving the quality of software maintenance. Therefore, an automated process of categorization and prioritization of bug reports is needed to address the aforementioned issues. Automated categorization and prioritization of bug reports have been explored recently by many researchers; however, limited progress has been made in this regard. In this research, we present a novel framework, titled CaPBug, for automated categorization and prioritization of bug reports. The framework is implemented using Natural Language Processing (NLP) and supervised machine learning algorithms. A baseline corpus is built with six categories and five prioritization levels by analyzing more than 2000 bug reports of Mozilla and Eclipse repository. Four classification algorithms i.e., Naive Bayes, Random Forest, Decision Tree, and Logistic Regression have been used to categorize and prioritize bug reports. We demonstrate that the CaPBug framework achieved an accuracy of 88.78% by using a Random Forest classifier with a textual feature for predicting the category. Similarly, using the CaPBug framework, an accuracy of 90.43% was achieved in predicting the priority of bug reports. Synthetic Minority Over-Sampling Technique (SMOTE) has been applied to address the class imbalance issue in priority classes.

show abstract

“…Y. Tan et al [37] proposed a novel approach for predicting severity. They linked the bug repositories post on stack overflows to the contents of Mozilla, Eclipse, and GCC bug reports.…”

Section: Volume XX 2020mentioning

confidence: 99%

CaPBug-A Framework for Automatic Bug Categorization and Prioritization Using NLP and Machine Learning Algorithms

2021

View full text Add to dashboard Cite

show abstract

“…There are models that predict bug severity using textual description in bug reports [7,8,9,10,11,12,13,14]. Reference [7] carried out a comparison between the newly proposed bug and already existing bugs in the bug database using BM25-based document similarity function and then assigned the severity for the new reports based on the similarity calculated.…”

Section: Related Workmentioning

confidence: 99%

“…Reference [9] compiled improved versions of bug reports of Mozilla, Eclipse and GCC by integrating questions and the correspond answers from stack trace. Subsequently, they trained Naï ve Bayesian, k-Nearest Neighbor algorithm (KNN), and Long Short-Term Memory (LSTM) models to classify bug reports into severity levels.…”

Section: Related Workmentioning

confidence: 99%

“…In such a skewed learning environment the testing sample may also contain the same distribution as the training sample. However, many of the existing bug severity prediction models trained from history data are nether mention the distribution of the datasets used to train the models, nor acknowledged this issue [7,8,9,10,11,12,13,14]. This implies that they assumed the underlying data distribution is balanced, which may not essentially be true in all real datasets such as I use in this project.…”

Section: Introductionmentioning

confidence: 98%

“…Hence, the objective of this project is to develop bug severity prediction models from imbalanced learning environments and test them properly using an appropriate metric. Many models developed in the literature [7,8,9,10,11,12,13,14] used the accuracy--correctly classified instances vs. the total number of instances presented--to evaluate the prediction quality in skewed learning environments [4,5]. However, the accuracy would not provide fair judgment on the quality of models trained from unevenly distributed datasets like used in this project.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Bug Severity Prediction using Keywords in Imbalanced Learning Environment

Ekanayake¹

2021

IJITCS

View full text Add to dashboard Cite

Reported bugs of software systems are classified into different severity levels before fixing them. The number of bug reports may not be equally distributed according to the severity levels of bugs. However, most of the severity prediction models developed in the literature assumed that the underlying data distribution is evenly distributed, which may not correct at all instances and hence, the aim of this study is to develop bug classification models from unevenly distributed datasets and tested them accordingly. To that end first, the topics or keywords of developer descriptions of bug reports are extracted using Rapid Keyword Extraction (RAKE) algorithm and then transferred them into numerical attributes, which combined with severity levels constructs datasets. These datasets are used to build classification models; Naïve Bayes, Logistic Regression, and Decision Tree Learner algorithms. The models’ prediction quality is measured using Area Under Recursive Operative Characteristics Curves (AUC) as the models learnt from more skewed environments. According to the results, the prediction quality of the Logistics Regression model is 0.65 AUC whereas the other two models recorded maximum 0.60 AUC. Though the datasets contain comparatively less number of instances from the high severity classes; Blocking and High, the Logistic Regression models predict the two classes with a decent AUC value of 0.65 AUC. Hence, this projects shows that the models can be trained from highly skewed datasets so that the models prediction quality is equally well over all the classes regardless of number of instances representing the class. Further, this project emphasizes that the models should be evaluated using the appropriate metrics when the models are trained from imbalance learning environments. Also, this work uncovers that the Logistic Regression model is also capable of classifying documents as Naïve Bayes, which is well known for this task.

show abstract

IM_NRFixer: A hybrid approach to alleviate class‐imbalance problem for predicting the fixability of Non‐Reproducible bugs

Goyal

Sardana

2020

J Software Evolu Process

View full text Add to dashboard Cite

Software maintenance is an important phase in the software development life cycle. Software projects maintain bug repositories to gather, organize, and keep track of bug reports. These bug reports are resolved by numerous software developers. Whenever the reported bug does not get resolved by the assigned developer, he marks the resolution of bug report as Non‐Reproducible (NR). When NR bugs are reconsidered, few of them get resolved, and their resolution changes from NR to fix (NRF). The main aim of this paper is to predict these fixable NRF bug reports. A major challenge in predicting NRF bugs from NR bugs is that only a small portion of NR bugs get fixed, i.e., class‐imbalance problem. For example, NRF bugs account for only 8.64%, 4.73 %, 4.56%, and 1.06% in NetBeans, Eclipse, Open Office, and Mozilla Firefox projects respectively. In this paper, we work on improving the classification performance on these imbalanced datasets. We propose IMNRFixer, a novel and hybrid NRF prediction tool. IMNRFixer uses three different techniques to combat class‐imbalance problem: undersampling, oversampling, and ensemble models. We evaluate the performance of IMNRFixer models on four large and open‐source projects of Bugzilla repository. Our results show that IMNRFixer outperforms conventional machine learning techniques. IMNRFixer achieves performance up to 71.7%, 93.1%, 91.7%, and 96.5% while predicting the minority class (NRF) for NetBeans, Eclipse, Open Office, and Mozilla Firefox projects, respectively.

show abstract

Bug severity prediction using question-and-answer pairs from Stack Overflow

Cited by 48 publications

References 8 publications

CaPBug-A Framework for Automatic Bug Categorization and Prioritization Using NLP and Machine Learning Algorithms

CaPBug-A Framework for Automatic Bug Categorization and Prioritization Using NLP and Machine Learning Algorithms

Bug Severity Prediction using Keywords in Imbalanced Learning Environment

IM_NRFixer: A hybrid approach to alleviate class‐imbalance problem for predicting the fixability of Non‐Reproducible bugs

Contact Info

Product

Resources

About

Bug severity prediction using question-and-answer pairs from Stack Overflow

Cited by 48 publications

References 8 publications

CaPBug-A Framework for Automatic Bug Categorization and Prioritization Using NLP and Machine Learning Algorithms

CaPBug-A Framework for Automatic Bug Categorization and Prioritization Using NLP and Machine Learning Algorithms

Bug Severity Prediction using Keywords in Imbalanced Learning Environment

IMNRFixer: A hybrid approach to alleviate class‐imbalance problem for predicting the fixability of Non‐Reproducible bugs

Contact Info

Product

Resources

About

IM_NRFixer: A hybrid approach to alleviate class‐imbalance problem for predicting the fixability of Non‐Reproducible bugs