Does calling structure information improve the accuracy of fault prediction?

Shin, Yonghee; Bell, Robert M.; Ostrand, Thomas J.; Weyuker, Elaine J.

doi:10.1109/msr.2009.5069481

Cited by 28 publications

(24 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Numerical, post-release fault prediction studies of other software products not related to Eclipse include (Bibi et al 2006;Kastro and Bener 2008;Khoshgoftaar and Munson 1990;Li et al 2006;Nagappan et al 2006;Ostrand et al 2004Ostrand et al , 2005Ostrand et al , 2010Bell et al 2006;Weyuker et al 2008;Shin et al 2009). Bibi et al (2006) compared twelve different models to determine the benefits of regression via classification.…”

Section: Numerical Prediction Of Post-release Software Faultsmentioning

confidence: 99%

“…The results showed that the individual developer's past performance was not an effective predictor of future bug locations. Shin et al (2009) used different combinations of LOC, static code metrics, change metrics, faults from previous releases, and calling structure information to construct negative binomial regression models. It appeared that the addition of calling structure information to a model based solely on non-calling structure code attributes provided noticeable improvement in prediction accuracy, but only marginally improved the best model based on history (i.e., change) and non-calling structure code attributes.…”

Section: Numerical Prediction Of Post-release Software Faultsmentioning

confidence: 99%

“…To quote Kutner et al (2004) "The fact that some or all predictor variables are correlated among themselves does not, in general, inhibit our ability to obtain a good fit nor does it tend to affect inferences about mean responses or predictions of new observations …" However, multicollinearity may cause the estimated regression coefficients to have a large sampling variability and thus affect explanatory studies. (Ohlsson and Alberg 1996;Fenton and Ohlsson 2000;Ostrand et al 2004Ostrand et al , 2005Bell et al 2006;Shin et al 2009). To justify the use of this metric for our data we explored how the post-release faults are distributed among Eclipse packages.…”

Section: Ntop20%mentioning

confidence: 99%

See 2 more Smart Citations

Assessment and cross-product prediction of software product line quality: accounting for reuse across products, over multiple releases

et al. 2014

View full text Add to dashboard Cite

The goals of cross-product reuse in a software product line (SPL) are to mitigate production costs and improve the quality. In addition to reuse across products, due to the evolutionary development process, a SPL also exhibits reuse across releases. In this paper, we empirically explore how the two types of reuse-reuse across products and reuse across releases-affect the quality of a SPL and our ability to accurately predict fault proneness. We measure the quality in terms of post-release faults and consider different levels of reuse across products (i.e., common, high-reuse variation, low-reuse variation, and single-use packages), over multiple releases. Assessment results showed that quality improved for common, low-reuse variation, and single-use packages as they evolved across releases. Surprisingly, within each release, among preexisting ('old') packages, the cross-product reuse did not affect the change and fault proneness. Cross-product predictions based on pre-release data accurately ranked the packages according to their post-release faults and predicted the 20 % most faulty packages. The predictions benefited from data available for other products in the product line, with models producing better results (1) when making predictions on smaller products (consisting mostly of common packages) rather than on larger products and (2) when trained on larger products rather than on smaller products.

show abstract

Section: Numerical Prediction Of Post-release Software Faultsmentioning

confidence: 99%

Section: Numerical Prediction Of Post-release Software Faultsmentioning

confidence: 99%

Section: Ntop20%mentioning

confidence: 99%

See 1 more Smart Citation

Assessment and cross-product prediction of software product line quality: accounting for reuse across products, over multiple releases

et al. 2014

View full text Add to dashboard Cite

show abstract

“…Schröter et al [17] showed that dependencies attributed to import statements in Java programs can be used to predict defects. Shin et al [18] and Naggappan and Ball [19] also used dependency information for defect prediction. Dependency relationships have been used by Zimmermann et al [20] to assess the quality of neighbouring entities.…”

Section: Introductionmentioning

confidence: 99%

Network Versus Code Metrics to Predict Defects: A Replication Study

Premraj¹,

Herzig

2011

2011 International Symposium on Empirical Software Engineering and Measurement

View full text Add to dashboard Cite

Abstract-Several defect prediction models have been proposed to identify which entities in a software system are likely to have defects before its release. This paper presents a replication of one such study conducted by Zimmermann and Nagappan [1] on Windows Server 2003 where the authors leveraged dependency relationships between software entities captured using social network metrics to predict whether they are likely to have defects. They found that network metrics perform significantly better than source code metrics at predicting defects. In order to corroborate the generality of their findings, we replicate their study on three open source Java projects, viz., JRuby, ArgoUML, and Eclipse. Our results are in agreement with the original study by Zimmermann and Nagappan when using a similar experimental setup as them (random sampling). However, when we evaluated the metrics using setups more suited for industrial use -forward-release and cross-project prediction -we found network metrics to offer no vantage over code metrics. Moreover, code metrics may be preferable to network metrics considering the data is easier to collect and we used only 8 code metrics compared to approximately 58 network metrics.

show abstract

“…D' Ambros et al (2009), Shin et al (2009), Nagappan et al (2010, and Madeyski and Jureczko (2015)). D 'Ambros et al (2009) specifically report that previous bug reports are the best predictors.…”

Section: Introductionmentioning

confidence: 99%

Software defect prediction: do different classifiers find the same defects?

2017

View full text Add to dashboard Cite

During the last 10 years, hundreds of different defect prediction models have been published. The performance of the classifiers used in these models is reported to be similar with models rarely performing above the predictive performance ceiling of about 80% recall. We investigate the individual defects that four classifiers predict and analyse the level of prediction uncertainty produced by these classifiers. We perform a sensitivity analysis to compare the performance of Random Forest, Naïve Bayes, RPart and SVM classifiers when predicting defects in NASA, open source and commercial datasets. The defect predictions that each classifier makes is captured in a confusion matrix and the prediction uncertainty of each classifier is compared. Despite similar predictive performance values for these four classifiers, each detects different sets of defects. Some classifiers are more consistent in predicting defects than others. Our results confirm that a unique subset of defects can be detected by specific classifiers. However, while some classifiers are consistent in the predictions they make, other classifiers vary in their predictions. Given our results, we conclude that classifier ensembles with decision-making strategies not based on majority voting are likely to perform best in defect prediction.

show abstract

Does calling structure information improve the accuracy of fault prediction?

Cited by 28 publications

References 24 publications

Assessment and cross-product prediction of software product line quality: accounting for reuse across products, over multiple releases

Assessment and cross-product prediction of software product line quality: accounting for reuse across products, over multiple releases

Network Versus Code Metrics to Predict Defects: A Replication Study

Software defect prediction: do different classifiers find the same defects?

Contact Info

Product

Resources

About