Can We Predict the Quality of Spectrum-based Fault Localization?

Golagha, Mojdeh; Pretschner, Alexander; Briand, Lionel C.

doi:10.1109/icst46399.2020.00012

Cited by 19 publications

(10 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our evaluation results in [8] show that the best model is based on random forest; combines 15 static code, dynamic execution, and test suite metrics; and shows an excellent discrimination power (AUC = 0.88). The most influential metrics are: four static metrics (% Methods with LoC>30, % Methods with Nesting Depth>5, % Methods with 3<=Nesting Depth<=5, Mean # of Fields per Type), four dynamic metrics (Mean Node Degree, Max.…”

Section: Predicting the Quality Of Sbflmentioning

confidence: 96%

How to Effectively Reduce Failure Analysis Time?

Golagha

2022

Ernst Denert Award for Software Engineering 2020

Self Cite

View full text Add to dashboard Cite

Debugging is one of the most expensive and challenging phases in the software development life-cycle. One important cost factor in the debugging process is the time required to analyze failures and find underlying faults. Two types of techniques that can help developers to reduce this analysis time are Failure Clustering and Automated Fault Localization. Although there is a plethora of these techniques in the literature, there are still some gaps that prevent their operationalization in real-world contexts. Besides, the abundance of these techniques confuses the developers in selecting a suitable method for their specific domain. In order to help developers in reducing analysis time, we propose methodologies and techniques that can be used standalone or in a form of a tool-chain. Utilizing this tool-chain, developers (1) know which data they need for further analysis, (2) are able to group failures based on their root causes, and (3) are able to find more information about the root causes of each failing group. Our tool-chain was initially developed based on state-of-the-art failure diagnosis techniques. We implemented and evaluated existing techniques. We built on and improved them where the results were promising and proposed new solutions where needed. The overarching goal of this study has been the applicability of techniques in practice.

show abstract

Section: Predicting the Quality Of Sbflmentioning

confidence: 96%

How to Effectively Reduce Failure Analysis Time?

Golagha

2022

Ernst Denert Award for Software Engineering 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…To determine how dissimilar two test cases are and, consequently, how likely they are to exercise the mutated statement with different values, we rely on widely adopted distance metrics. In the case of S1, we rely on the Jaccard and Ochiai indices, which are two similarity indices for binary data which have been successfully used to compare program executions based on code coverage [99], [100], [101]. Given two test cases T A and T B , the Jaccard (D J ) and Ochiai (D O ) distances are computed as follows:…”

Section: Step 6: Test Prioritizationmentioning

confidence: 99%

Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results in the Space Domain

Olivares

Pastore

Briand

2022

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

On-board embedded software developed for spaceflight systems (space software) must adhere to stringent software quality assurance procedures. For example, verification and validation activities are typically performed and assessed by third party organizations. To further minimize the risk of human mistakes, space agencies, such as the European Space Agency (ESA), are looking for automated solutions for the assessment of software testing activities, which play a crucial role in this context. Though space software is our focus here, it should be noted that such software shares the above considerations, to a large extent, with embedded software in many other types of cyber-physical systems. Over the years, mutation analysis has shown to be a promising solution for the automated assessment of test suites; it consists of measuring the quality of a test suite in terms of the percentage of injected faults leading to a test failure. A number of optimization techniques, addressing scalability and accuracy problems, have been proposed to facilitate the industrial adoption of mutation analysis. However, to date, two major problems prevent space agencies from enforcing mutation analysis in space software development. First, there is uncertainty regarding the feasibility of applying mutation analysis optimization techniques in their context. Second, most of the existing techniques either can break the real-time requirements common in embedded software or cannot be applied when the software is tested in Software Validation Facilities, including CPU emulators and sensor simulators. In this paper, we enhance mutation analysis optimization techniques to enable their applicability to embedded software and propose a pipeline that successfully integrates them to address scalability and accuracy issues in this context, as described above. Further, we report on the largest study involving embedded software systems in the mutation analysis literature. Our research is part of a research project funded by ESA ESTEC involving private companies (GomSpace Luxembourg and LuxSpace) in the space sector. These industry partners provided the case studies reported in this paper; they include an on-board software system managing a microsatellite currently on-orbit, a set of libraries used in deployed cubesats, and a mathematical library certified by ESA.

show abstract

“…After obtaining the necessary information, SBFL utilizes various formulas to compute the suspiciousness of program statements and produces a ranking list to locate the fault. Golagha et al [15] proposed the model to assess the potential effectiveness of fault localization. Simalarly, Dutta et al [16] proposed a modified Fisher's test-based statistical method that maked use of test execution results as well as statement coverage information to determine the suspiciousness of executable statements.…”

Section: Spectrum-based Fault Localization (Sbfl)mentioning

confidence: 99%

An Innovative Framework for Coincidental Correctness Impacting on Fault Localization

Cao

Sun

2022

Symmetry

View full text Add to dashboard Cite

An important research aspect of Spectrum-Based Fault Localization (SBFL) is the influence factors of the effectiveness of suspiciousness formulas from the perspective of symmetry. Coincidental correctness is one of the most important factors impacting the effectiveness of suspiciousness formulas. The influence of fault localization by coincidental correctness has attracted a large amount of research in the perspective of empirical study; however, it can hardly be considered as sufficiently comprehensive when there are a large number of the symmetrical suspiciousness formulas. Therefore, we first develop an innovative theoretical framework with function derivation investigating suspiciousness formulas impacted by coincidental correctness. We define three types of relations between formulas affected by coincidental correctness: namely, improved type, invariant type and uncertain type. We investigated 30 suspiciousness formulas using this framework and group them into three categories. Furthermore, we conduct an empirical study to verify the effectiveness of SBFL affected by coincidental correctness on four relatively large C programs. We proved that coincidental correctness has a positive effect on 23 out of these 30 formulas, no effect on 5 of them, and the effect on the remaining 2 of them depend on certain conditions. The experimental results show that the effectiveness of some suspiciousness formulas can be enhanced and that of some suspiciousness formulas remain unchanged.

show abstract

Can We Predict the Quality of Spectrum-based Fault Localization?

Cited by 19 publications

References 39 publications

How to Effectively Reduce Failure Analysis Time?

How to Effectively Reduce Failure Analysis Time?

Mutation Analysis for Cyber-Physical Systems: Scalable Solutions and Results in the Space Domain

An Innovative Framework for Coincidental Correctness Impacting on Fault Localization

Contact Info

Product

Resources

About