HDA: Cross-Project Defect Prediction via Heterogeneous Domain Adaptation With Dictionary Learning

Zhou, Xu; Yuan, Peipei; Zhang, Tao; Tang, Yutian; Li, Shuai; Xia, Zhen

doi:10.1109/access.2018.2873755

Cited by 30 publications

(14 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this method does not resolve the class imbalance problem before applying transfer learning. Xu et al [11] developed heterogeneous domain adaptation with dictionary learning for HDP. This method employed the domain adaptation method to insert the data from the two projects and then measured the difference between them using dictionary learning to predict the defects.…”

Section: Related Workmentioning

confidence: 99%

Improved SMOTE and Optimized Siamese Neural Networks for Class Imbalanced Heterogeneous Cross Project Defect Prediction

2022

IJIES

View full text Add to dashboard Cite

Heterogeneous Cross-Project Defect Prediction (HCPDP) predicts the defects by incorporating the project datasets with different distribution and metrics. However, the class imbalance problem and shallow feature learning of classifiers reduce the overall performance. This paper presents an HCPDP using Improved Synthetic Minority Oversampling Technique (ISMOTE) and hybrid deep learning classifier of Golden Eagle Optimized Siamese Neural Networks (GEO-SNN). Initially, Peters filter removes irrelevant instances in source-target dataset based on heterogeneous metrics. ISMOTE, enhanced through fuzzy mutual information based KNN attribute weights assignment, is applied solve the class imbalance problem. Features are scaled using Z-Score standardization and metric matching by Spearman's Rank Correlation (SRC). Finally, GEO-SNN, developed by SNN parameter optimization using GEO, performs deep semantic feature learning for defect prediction. Experiments using benchmark datasets showed that the proposed ISMOTE and GEO-SNN based HCPDP technique has provided 99% defect prediction accuracy and reduced processing time by 20%than the state-of-the-art methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Improved SMOTE and Optimized Siamese Neural Networks for Class Imbalanced Heterogeneous Cross Project Defect Prediction

2022

IJIES

View full text Add to dashboard Cite

show abstract

“…Then, we should investigate more advanced imbalance learning methods since the class imbalance problem is more serious in SVP when compared to SDP [18]. Except for the withinproject SVP scenario considered in our study, the cross-project SVP scenario [49,50] also needs investigation. Cross-project SVP denotes that SVP models are constructed from one project and then used to predict potentially vulnerable modules in another project.…”

Section: Related Workmentioning

confidence: 99%

Empirical studies on the impact of filter‐based ranking feature selection on security vulnerability prediction

et al. 2020

View full text Add to dashboard Cite

Security vulnerability prediction (SVP) can construct models to identify potentially vulnerable program modules via machine learning. Two kinds of features from different points of view are used to measure the extracted modules in previous studies. One kind considers traditional software metrics as features, and the other kind uses text mining to extract term vectors as features. Therefore, gathered SVP data sets often have numerous features and result in the curse of dimensionality. In this article, we mainly investigate the impact of filter-based ranking feature selection (FRFS) methods on SVP, since other types of feature selection methods have too much computational cost. In empirical studies, we first consider three real-world large-scale web applications. Then we consider seven methods from three FRFS categories for FRFS and use a random forest classifier to construct SVP models. Final results show that given the similar code inspection cost, using FRFS can improve the performance of SVP when compared with state-of-the-art baselines. Moreover, we use McNemar's test to perform diversity analysis on identified vulnerable modules by using different FRFS methods, and we are surprised to find that almost all the FRFS methods can identify similar vulnerable modules via diversity analysis.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

“…In recent years, many other supervised and unsupervised models (e.g., CCUM [14], TLEL [19], and MULTI [20]) have been proposed to push forward the state of the arts. Fukushima et al [47] found given a project with a few historical changes, we could use the changes from other projects as the training data to build the prediction model, as followed by other research [48], [49]. More recently, some study [50], [51] focuses on the impact of classification techniques, and some other study [52]- [57] attempts to handle the problem of imbalanced data in effort-aware JIT-SDP.…”

Section: A Effort-aware Jit-sdpmentioning

confidence: 99%

Automatic Feature Exploration and an Application in Defect Prediction

Qiu

Liu

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Many software engineering tasks heavily rely on hand-crafted software features, e.g., defect prediction, vulnerability discovery, software requirements, code review, and malware detection. Previous solutions to these tasks usually directly use the hand-crafted features or feature selection techniques for classification or regression, which usually leads to suboptimal results due to their lack of powerful representations of the hand-crafted features. To address the above problem, in this paper, we adopt the effortaware just-in-time software defect prediction (JIT-SDP), which is a typical hand-crafted-feature-based task, as an example, to exploit new possible solutions. We propose a new model, named neural forest (NF), which uses the deep neural network and decision forest to build a holistic system for the automatic exploration of powerful feature representations that are used for the following classification. NF first employs a deep neural network to learn new feature representations from hand-crafted features. Then, a decision forest is connected after the neural network to perform classification and in the meantime, to guide the learning of feature representation. NF mainly aims at solving the challenging problem of combining the two different worlds of neural networks and decision forests in an end-to-end manner. When compared with previous state-of-the-art defect predictors and five designed baselines on six well-known benchmarks for within-and cross-project defect prediction, NF achieves significantly better results. The proposed NF model is generic to the classification problems which rely on the hand-crafted features.INDEX TERMS Feature exploration, hand-crafted features, defect prediction.

show abstract

HDA: Cross-Project Defect Prediction via Heterogeneous Domain Adaptation With Dictionary Learning

Cited by 30 publications

References 59 publications

Improved SMOTE and Optimized Siamese Neural Networks for Class Imbalanced Heterogeneous Cross Project Defect Prediction

Improved SMOTE and Optimized Siamese Neural Networks for Class Imbalanced Heterogeneous Cross Project Defect Prediction

Empirical studies on the impact of filter‐based ranking feature selection on security vulnerability prediction

Automatic Feature Exploration and an Application in Defect Prediction

Contact Info

Product

Resources

About