Correlation feature and instance weights transfer learning for cross project software defect prediction

Abstract

Due to the differentiation between training and testing data in the feature space, crossproject defect prediction (CPDP) remains unaddressed within the field of traditional machine learning. Recently, transfer learning has become a research hot-spot for building classifiers in the target domain using the data from the related source domains. To implement better CPDP models, recent studies focus on either feature transferring or instance transferring to weaken the impact of irrelevant cross-project data. Instead, this work proposes a dual weighting mechanism to aid the learning process, considering both feature transferring and instance transferring. In our method, a local data gravitation between source and target domains determines instance weight, while features that are highly correlated with the learning task, uncorrelated with other features and minimizing the difference between the domains are rewarded with a higher feature weight. Experiments on 25 real-world datasets indicate that the proposed approach outperforms the existing CPDP methods in most cases. By assigning weights based on the different contribution of features and instances to the predictor, the proposed approach is able to build a better CPDP model and demonstrates substantial improvements over the stateof-the-art CPDP models.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Revisiting ‘revisiting supervised methods for effort‐aware cross‐project defect prediction’

“…Zou et al. [26] proposed a CPDP method called CFIW‐TNB, which took into consideration instance‐transfer learning and feature‐transfer learning. Li et al.…”

“…Most previous EADP studies [13,[16][17][18][19][20][21] usually built predictive models on historical labelled software modules and then predicted the defect-proneness of unlabelled modules in the same project, referred to as within-project EADP. However, in actual software development scenarios, it is hard to obtain a large amount of historical data from the same project, especially for newly developed software [22][23][24][25][26][27]. Therefore, Ni et al [28] proposed an effort-aware cross-project defect prediction (EACPDP) method called EASC in their IEEE transactions on software engineering (TSE) paper titled 'Revisiting supervised and unsupervised methods for effort-aware cross-project defect prediction'.…”

“…Wu et al [81] proposed a cost-sensitive kernelised semi-supervised dictionary learning method for CPDP, which can use limited labelled defective modules and lots of unlabelled modules in the kernel space and consider the misclassification cost during dictionary learning. Zou et al [26] proposed a CPDP method called CFIW-TNB, which took into consideration instance-transfer learning and feature-transfer learning. Li et al [24] proposed a selection-based kernelised discriminant subspace alignment method for CPDP, which took into account data distribution differences, non-linearity, and discriminant problems at the same time.…”

An Approach to Improving Homogeneous Cross-Project Defect Prediction by Jensen-Shannon Divergence and Relative Density

“…Gong et al [47] utilized the thought of stratification embedded in the nearest neighbor to produce evolving training datasets with balanced data. Zou et al [48] proposed a method named Joint Feature representation with double marginalized denoising auto-encoders to learn the global and local features, and they introduced local data gravitation between source and target domains to determine instance weight in the learning process (Zou et al [49]). Jin et al [50] used two support vector machines to implement domain adaptation to match data distribution.…”

Using Cost-cognitive Bagging Ensemble to Improve Cross-project Defects Prediction

“…Transfer learning addresses these issues by transferring knowledge extracted from the source projects to the target project for building more accurate CPDP models [13]. The CPDP approaches based on transfer learning can be classified as either feature-based or instance-based approaches [14][15].…”