Reproducibility and replicability of software defect prediction studies

Mahmood, Zaheed; Bowes, David; Hall, Tracy; Lane, Peter C. R.; Petrić, Jean

doi:10.1016/j.infsof.2018.02.003

Cited by 26 publications

(15 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, the fault tolerance is low and the system availability is reduced during the rejuvenation [30,31]. 6. One of the problems in optimizing software rejuvenation is to select a suitable server for web application allocation that can reduce fail frequency.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Assuming a probability of 50%, selection of either the shrinking encircling movement or the spiral model movement is simulated during iterations of the algorithm as in Eq. (6). Here p is a random number in…”

Section: Spiral Updating Positionmentioning

confidence: 99%

“…In previous works, in order to improve software reliability, software defect prediction was applied to the process of software maintenance to identify potential bugs [6]. Li et al [7] suggested a hybrid method called the software process risk optimization method that integrates risk management into the software process model.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Dynamic software rejuvenation in web services: a whale optimizationalgorithm-based approach

Kalantari¹,

Ebrahimnejad²,

Motameni³

2020

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

In this paper, we suggest a method for determining the restarting time for web services to increase availability, known as rejuvenation. We consider different parameters such as number of users, maximum service request number, response time, and throughput of a web service to determine its restarting time. Software rejuvenation is an effective technique to counteract software aging in continuously running applications such as web service-based systems. In these systems, web services are allocated based on the needs of the receivers and facilities of servers. One of the challenges while assigning web services is selecting the appropriate server to reduce faults. Since the selection of a server among candidates while maintaining the optimal quality of service is an NP-hard problem, metaheuristics seem to be suitable. In this paper, we propose dynamic software rejuvenation as a proactive fault-tolerance technique based on the whale optimization algorithm. The threshold for the rejuvenation of each of the web services is considered and training is done based on the features of the service providers as well as the needs of the receivers. The whale optimization algorithm with the criterion of movement radius is utilized for flexibility of web service provider selection. Here, we detect and rejuvenate systems that required rejuvenation before the occurrence of a fault. The simulation results reveal that our strategy can decrease the failure rate by an average of 30 percent in comparison with state-of-the-art strategies and improve the system availability in web services.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Spiral Updating Positionmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic software rejuvenation in web services: a whale optimizationalgorithm-based approach

Kalantari¹,

Ebrahimnejad²,

Motameni³

2020

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

show abstract

“…proposed Cluster-based Over-sampling with noise filtering (KMFOS) approach to handle class imbalance problem in the context of Software Defect Prediction [21]. Mahmood et al, conducted a study of replicability as well as reproducibility of defect prediction in research areas to show the importance of any topic behind the replicability [12]. Tong et al worked on SDP using dual-stage ensembles and encoders [16].…”

Section: Literature Surveymentioning

confidence: 99%

Software Defect Prediction u sing Support Vectorised Data and Intelligent Techniques

Kumar*,

Prasad

2020

IJITEE

View full text Add to dashboard Cite

Software enhances the working capability of any business. Developing such a software entrusts the developing organization to build defect free software. In this context we have used PC1 dataset(NASA dataset) which has sufficient parameters for analysis. Intelligent techniques using different methodologies have been applied exhaustively on the PC1 data to find out the best intelligent technique for software defect. As the PC1 data is highly imbalanced data, there was biasness in the prediction of the intelligent techniques. Hence, to overcome this issue, in this paper we tried to propose best balancing method along with the intelligent technique to predict the software defect accurately.

show abstract

“…Apart from effectiveness, replicability and reproducibility are also widely accepted as important considerations in scientific research [12,13,63,101,114]. Commonly, replicability refers to whether a reported experimental result can be exactly reproduced with the same model and the same data.…”

Section: Introductionmentioning

confidence: 99%

On the Replicability and Reproducibility of Deep Learning in Software Engineering

Liu,

Gao,

Xia

et al. 2020

Preprint

View full text Add to dashboard Cite

Deep learning (DL) techniques have gained significant popularity among software engineering (SE) researchers in recent years. This is because they can often solve many SE challenges without enormous manual feature engineering effort and complex domain knowledge. Although many DL studies have reported substantial advantages over other state-of-the-art models on effectiveness, they often ignore two factors: (1) replicability -whether the reported experimental result can be approximately reproduced in high probability with the same DL model and the same data; and (2) reproducibility -whether one reported experimental findings can be reproduced by new experiments with the same experimental protocol and DL model, but different sampled real-world data. Unlike traditional machine learning (ML) models, DL studies commonly overlook these two factors and declare them as minor threats or leave them for future work. This is mainly due to high model complexity with many manually set parameters and the time-consuming optimization process. In this study, we conducted a literature review on 93 DL studies recently published in twenty SE journals or conferences. Our statistics show the urgency of investigating these two factors in SE, where only 10.8% of the studies discussed any research questions affecting replicability and/or reproducibility. More than 74.2% of the studies do not even share source code and data to support the replicability of their complex models. Moreover, we re-ran four representative DL models in SE. Experimental results show the importance of replicability and reproducibility, where the reported performance of a DL model could not be replicated for an unstable optimization process. Reproducibility could be substantially compromised if the model training is not convergent, or if performance is sensitive to the size of vocabulary and testing data. It is therefore urgent for the SE community to provide a long-lasting link to a replication package, enhance DL-based solution stability and convergence, and avoid performance sensitivity on different sampled data.CCS Concepts: • Software and its engineering → Software maintenance tools.

show abstract

Reproducibility and replicability of software defect prediction studies

Cited by 26 publications

References 40 publications

Dynamic software rejuvenation in web services: a whale optimizationalgorithm-based approach

Dynamic software rejuvenation in web services: a whale optimizationalgorithm-based approach

Software Defect Prediction u sing Support Vectorised Data and Intelligent Techniques

On the Replicability and Reproducibility of Deep Learning in Software Engineering

Contact Info

Product

Resources

About