Developing fault predictors for evolving software systems

Nikora, Allen P.; Munson, John C.

doi:10.1109/metric.2003.1232479

Cited by 36 publications

(37 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Widely used: Many researchers use static features to guide software quality predictions (see [41,43,44,45,46,47,48,49,50,51,52,53,54]). Verification and validation (V&V) textbooks advise using static code features to select modules worthy of manual inspections [55].…”

Section: On Defect Prediction Using Static Code Featuresmentioning

confidence: 99%

On the relative value of cross-company and within-company data for defect prediction

et al. 2009

View full text Add to dashboard Cite

We propose a practical defect prediction approach for companies that do not track defect related data. Specifically, we investigate the applicability of cross-company (CC) data for building localized defect predictors using static code features.Firstly, we analyze the conditions, where CC data can be used as is. These conditions turn out to be quite few. Then we apply principles of analogy-based learning (i.e. nearest neighbor (NN) filtering) to CC data, in order to fine tune these models for localization. We compare the performance of these models with that of defect predictors learned from within-company (WC) data. As expected, we observe that defect predictors learned from WC data outperform the ones learned from CC data. However, our analyses also yield defect predictors learned from NN-filtered CC data, with performance close to, but still not better than, WC data. Therefore, we perform a final analysis for determining the minimum number of local defect reports in order to learn WC defect predictors. We demonstrate in this paper that the minimum number of data samples required to build effective defect predictors can be quite small and can be collected quickly within a few months.Hence, for companies with no local defect data, we recommend a two-phase approach that allows them to employ the defect prediction process instantaneously. In phase one, companies should use NN-filtered CC data to initiate the defect prediction process and simultaneously start collecting WC (local) data. Once enough WC data is collected (i.e. after a few months), organizations should switch to phase two and use predictors learned from WC data.

show abstract

Section: On Defect Prediction Using Static Code Featuresmentioning

confidence: 99%

On the relative value of cross-company and within-company data for defect prediction

et al. 2009

View full text Add to dashboard Cite

show abstract

“…Na- gappan and Ball (2005a, 2005b),Hall and Munson (2000),Nikora and Munson (2003),Khoshgoftaar (2001),Tang and Khoshgoftaar (2004),Khoshgoftaar and Seliya (2003),Porter and Selby (1990),Tian and Zelkowitz (1995),Khoshgoftaar and Allen (2001),Srinivasan and Fisher (1995).…”

mentioning

confidence: 99%

Defect prediction from static code features: current results, limitations, new approaches

et al. 2010

View full text Add to dashboard Cite

Building quality software is expensive and software quality assurance (QA) budgets are limited. Data miners can learn defect predictors from static code features which can be used to control QA resources; e.g. to focus on the parts of the code predicted to be more defective.Recent results show that better data mining technology is not leading to better defect predictors. We hypothesize that we have reached the limits of the standard learning goal of maximizing area under the curve (AUC) of the probability of false alarms and probability of detection "AUC(pd, pf)"; i.e. the area under the curve of a probability of false alarm versus probability of detection.Accordingly, we explore changing the standard goal. Learners that maximize "AUC(effort, pd)" find the smallest set of modules that contain the most errors. Autom Softw Eng (2010) 17: 375-407 WHICH is a meta-learner framework that can be quickly customized to different goals. When customized to AUC(effort, pd), WHICH out-performs all the data mining methods studied here. More importantly, measured in terms of this new goal, certain widely used learners perform much worse than simple manual methods.Hence, we advise against the indiscriminate use of learners. Learners must be chosen and customized to the goal at hand. With the right architecture (e.g. WHICH), tuning a learner to specific local business goals can be a simple task.

show abstract

“…Finally, there are errors that are introduced directly into the source. Nikora and Munson developed a standard for the enumeration of faults based on the structural characteristics of the MDS software system [9]. Changes to the system are visible at the module level (i.e.…”

Section: Related Workmentioning

confidence: 99%

EQ-Mine: Predicting Short-Term Defects for Software Evolution

Ratzinger

Pinzger

Gall

Fundamental Approaches to Software Engineering

View full text Add to dashboard Cite

Abstract. We use 63 features extracted from sources such as versioning and issue tracking systems to predict defects in short time frames of two months. Our multivariate approach covers aspects of software projects such as size, team structure, process orientation, complexity of existing solution, difficulty of problem, coupling aspects, time constrains, and testing data. We investigate the predictability of several severities of defects in software projects. Are defects with high severity difficult to predict? Are prediction models for defects that are discovered by internal staff similar to models for defects reported from the field?We present both an exact numerical prediction of future defect numbers based on regression models as well as a classification of software components as defect-prone based on the C4.5 decision tree. We create models to accurately predict short-term defects in a study of 5 applications composed of more than 8.000 classes and 700.000 lines of code. The model quality is assessed based on 10-fold cross validation.

show abstract

Developing fault predictors for evolving software systems

Cited by 36 publications

References 14 publications

On the relative value of cross-company and within-company data for defect prediction

On the relative value of cross-company and within-company data for defect prediction

Defect prediction from static code features: current results, limitations, new approaches

EQ-Mine: Predicting Short-Term Defects for Software Evolution

Contact Info

Product

Resources

About