How Far We Have Progressed in the Journey? An Examination of Cross-Project Defect Prediction

Zhou, Yuming; Yang, Yibiao; Lü, Hongmin; Chen, Lin; Li, Yanhui; Zhao, Yangyang; Qian, Junyan; Xu, Bin

doi:10.1145/3183339

Cited by 164 publications

(103 citation statements)

References 111 publications

Supporting

Mentioning

101

Contrasting

Unclassified

Order By: Relevance

“…In the experiments, we use three widely used datasets: ReLink, AEEEM, and MORPH. Table shows the characteristics of these datasets.…”

Section: Experiments Design and Results Analysismentioning

confidence: 99%

“…Threats to the external validity of our study are that the observed experimental results may not be applicable to other software projects. To guarantee the representative of experimental subjects, we choose the three extensively used datasets AEEEM, MOPRH, and ReLink …”

Section: Experiments Design and Results Analysismentioning

confidence: 99%

“…MTL is such an ideal candidate technique, which treats related projects simultaneously by extracting and utilizing appropriate shared information among them. On the contrary, previous studies simply treated these projects independently and ignored the relatedness among projects, which can be called as single‐task learning (STL). Figure illustrates the difference between STL and MTL.…”

Section: Motivation and Backgroundmentioning

confidence: 99%

“…Different from previous studies which treat each project independently, in this paper, we treat all these projects fairly and simultaneously.…”

Section: Motivation and Backgroundmentioning

confidence: 99%

“…Previous approaches can build one defect prediction model for one target project at a time, while mask extended from a previous study can build multiple defect prediction models for multiple target projects at a time. Figure presents the overall framework of mask .…”

Section: Our Proposed Mask Approachmentioning

confidence: 99%

See 4 more Smart Citations

Multitask defect prediction

Chen

Xia

et al. 2019

J Software Evolu Process

View full text Add to dashboard Cite

Within‐project defect prediction assumes that we have sufficient labeled data from the same project, while cross‐project defect prediction assumes that we have plenty of labeled data from source projects. However, in practice, we might only have limited labeled data from both the source and target projects in some scenarios. In this paper, we want to apply multitask learning to investigate such a new scenario. To our best knowledge, this problem (ie, both the source project and the target project have limited labeled data) has not been thoroughly investigated, and we are the first to propose a novel multitask defect prediction approach mask. mask consists of a differential evolution optimization phase and a multitask learning phase. The former phase aims to find optimal weights for shared and nonshared information in related projects (ie, the target project and its related source projects), while the latter phase builds prediction models for each project simultaneously. To verify the effectiveness of mask, we perform experimental studies on 18 real‐world software projects and compare our approach with four state‐of‐the‐art baseline approaches: single‐task learning (STL), simple combined learning (SCL), Peters filter, and Burak filter. Experimental results show that mask can achieve F1 of 0.397 and AUC of 0.608 on average with a few labeled data (ie, 10% of data). Across the 18 projects, mask can outperform baseline methods significantly in terms of F1 and AUC. Therefore, by utilizing the relatedness among multiple projects, mask can perform significantly better than the state‐of‐the‐art methods. The results confirm that mask is promising for software defect prediction when the source and target projects both have limited training data.

show abstract

“…In the experiments, we use three widely used datasets: ReLink, AEEEM, and MORPH. Table shows the characteristics of these datasets.…”

Section: Experiments Design and Results Analysismentioning

confidence: 99%

Section: Experiments Design and Results Analysismentioning

confidence: 99%

Section: Motivation and Backgroundmentioning

confidence: 99%

“…Different from previous studies which treat each project independently, in this paper, we treat all these projects fairly and simultaneously.…”

Section: Motivation and Backgroundmentioning

confidence: 99%

Section: Our Proposed Mask Approachmentioning

confidence: 99%

See 3 more Smart Citations

Multitask defect prediction

Chen

Xia

et al. 2019

J Software Evolu Process

View full text Add to dashboard Cite

show abstract

A novel software defect prediction approach using modified objective cluster analysis

Ren¹,

Zhang

2020

Concurrency and Computation

View full text Add to dashboard Cite

A novel approach for software defect prediction of unlabeled datasets is proposed using modified objective cluster analysis (OCA). In this approach, the first step is to construct the distance matrix of instances in the datasets by utilizing the automatically determined clusters through the modified OCA. Then the dipoles within different instances are categorized into two different groups. Finally, the clusters of instances are produced, and software defects can be predicted by imposing a modified consistency criterion. Case study and comparative experiments were conducted based on 12 public datasets selected from the databases of Promise and ReLink using multiple different unsupervised algorithms and cross‐project approaches. There are two experimental settings: experiments with datasets that contain all metrics and experiments with datasets that contain only module size metrics. The results were evaluated by the index of precision, recall, F‐measure, and receiver operating characteristic curve (AUC). Furthermore, a complexity analysis of the tested algorithms was conducted as well. In experiments with datasets with all metrics, the proposed OCA gets the best results in four indexes, and the average values of precision, recall, F‐measure, and AUC were improved by a minimum of 1.52%, 2.78%, 19.84%, and 0.93%, respectively. In experiments with datasets with only module size metrics, the proposed OCA performed the best results in four indexes also, and the average values of precision, F‐measure, and AUC were improved by a minimum of 8.8%, 2.59%, and 8.36%, respectively. The proposed algorithm is of low complexity and provides a new way to efficiently predict software defects with unlabeled datasets.

show abstract

Cross‐project defect prediction method based on genetic algorithm feature selection

Zhu

2023

Engineering Reports

View full text Add to dashboard Cite

With the continuous development of Internet technology, the role of software in life is increasing, and software defect prediction (SDP) is a key means to ensure software reliability. SDP is to predict the modules that may have defects in advance based on the historical data of software projects, and its purpose is to maximize the use of testing resources. However, in the actual development process, the project that needs to be predicted is often a new project for which there is little or no historical data. Therefore, how to use the massive data of other related projects to build a cross‐project software defect prediction (CPDP) model has received extensive attention from scholars. However, due to the differences in data distribution and class imbalance between different projects, the performance of CPDP is greatly affected. Therefore, on the basis of CPDP, this article proposes a feature selection method based on genetic algorithm (genetic algorithm feature selection, GAFS). GAFS mainly includes two stages: feature selection and ensemble training. In the feature selection stage, this article proposes a global search adaptive feature selection method based on genetic algorithm, which uses the integrated training results of candidate feature subsets on target data to migrate the optimal feature subset. In the ensemble training phase, the EasyEnsemble method is used to alleviate the class imbalance problem, multiple naive Bayesian classifiers are constructed, and then the final model is constructed through ensemble learning. In this article, F1‐score and MCC are used as the test indicator, and comparative experiments are carried out on AEEEM and Promise. The results show that compared with the five comparison methods, GAFS can improve the average F1‐score and MCC much more. For example, GAFS can improve the average F1‐score value by 38.9%, 31.6%, 35.1%, 22.0%, and 31.6%, respectively. In most cases, it can effectively improve the performance of the model and achieve better prediction results.

show abstract

How Far We Have Progressed in the Journey? An Examination of Cross-Project Defect Prediction

Cited by 164 publications

References 111 publications

Multitask defect prediction

Multitask defect prediction

A novel software defect prediction approach using modified objective cluster analysis

Cross‐project defect prediction method based on genetic algorithm feature selection

Contact Info

Product

Resources

About