An evaluation of pure spectrum‐based fault localization techniques for large‐scale software systems

Heiden, Simon; Grunske, Lars; Kehrer, Timo; Keller, Fabian; Hoorn, André van; Filieri, Antonio; Lo, David

doi:10.1002/spe.2703

Cited by 31 publications

(13 citation statements)

References 80 publications

(159 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, SBFL cannot distinguish between program elements that exhibit the same execution patterns. The reason behind this issue is that SBFL techniques leverage hit spectra (i.e., whether an element is executed or not) only as the abstraction for program executions without considering any other useful contextual information [147]. In other words, they represent a program's behavior as an abstract hit spectra model that cannot capture the semantics of each program element individually [44].…”

Section: R No Contextual Informationmentioning

confidence: 99%

A Survey of Challenges in Spectrum-Based Software Fault Localization

Sarhan

Beszedes

2022

IEEE Access

View full text Add to dashboard Cite

In software debugging, fault localization is the most difficult, expensive, tedious, and timeconsuming task, particularly for large-scale software systems. This is due to the fact that it requires significant human participation and it is difficult to automate its sub-tasks. Therefore, there is a high demand for automatic fault localization techniques that can help software engineers effectively find the locations of faults with minimal human intervention. This has led to the proposal of implementing different types of such techniques. However, Spectrum Based Fault Localization (SBFL) is considered amongst the most prominent techniques in this respect due to its efficiency and effectiveness. In SBFL, the probability of each program element (e.g., statement, block, or function) being faulty is calculated based on the results of executing test cases and their corresponding code coverage information. However, SBFL techniques are not yet widely adopted in the industry. The rationale behind this is that they pose a number of issues and their performance is affected by several influential factors. For example, the characteristics of bugs, target programs, test suites, and supporting tools make their effectiveness differ dramatically from one case to another. There are massive studies on SBFL that cover its usage, formulas, performance, etc. So far, no dedicated survey points out comprehensively the issues of SBFL. In this paper, various SBFL challenges and issues have been identified, categorized, and discussed alongside many directions. Also, the paper raises awareness of the works being achieved to address the identified issues and suggests some potential solutions too.INDEX TERMS Program spectra, spectrum based fault localization, software testing, challenges and issues, survey.

show abstract

Section: R No Contextual Informationmentioning

confidence: 99%

A Survey of Challenges in Spectrum-Based Software Fault Localization

Sarhan

Beszedes

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Elements with high scores are placed at the beginning of the list, and elements with low scores are placed at the bottom. Heiden et al (2019) argue that the SBFL technique only exploits the program spectrum as an abstraction for program execution without considering any other useful contextual information. Vancsics et al (2021) addresses this problem by using method call frequency.…”

Section: Related Workmentioning

confidence: 99%

The improved dynamic slicing for spectrum-based fault localization

Cao

Wang

Deng

et al. 2022

PeerJ Computer Science

View full text Add to dashboard Cite

Background Spectrum-based Fault localization have proven to be useful in the process of software testing and debugging. However, how to improve the effectiveness of software fault localization has always been a research hot spot in the field of software engineering. Dynamic slicing can extract program dependencies under certain conditions. Thus, this technology is expected to benefit for locating fault. Methods We propose an improved dynamic slicing for spectrum-based fault localization under a general framework. We first obtain the dynamic slice of program execution. Secondly, we construct a mixed slice spectrum matrix from the dynamic slice of each test case and the corresponding test results. Finally, we compute the suspiciousness value of each statement in the mixed-slice spectram matrix. Results To verify the performance of our method, we conduct an empirical study on 15 widely used open-source programs. Experimental results show that our approach achieves significant improvement than the compared techniques. Conclusions Our approach can reduce approximately 1% to 17.79% of the average cost of code examined significantly.

show abstract

“…Notably, their evaluation results shows that results on artificial faults do not predict results on real faults for both techniques, and a hybrid technique is significantly better than both techniques. Keller et al (2017) and Heiden et al (2019) evaluated the effectiveness of statistical fault localization on real world large-scale software systems. The authors found that, for realistic large-scale programs, the accuracy of statistical debugging is not suitable for human developers.…”

Section: Evaluation Of Fault Localization Techniquesmentioning

confidence: 99%

Locating Faults with Program Slicing: An Empirical Analysis

Soremekun¹,

Kirschner²,

Böhme³

et al. 2021

Preprint

View full text Add to dashboard Cite

Statistical fault localization is an easily deployed technique for quickly determining candidates for faulty code locations. If a human programmer has to search the fault beyond the top candidate locations, though, more traditional techniques of following dependencies along dynamic slices may be better suited. In a large study of 457 bugs (369 single faults and 88 multiple faults) in 46 open source C programs, we compare the effectiveness of statistical fault localization against dynamic slicing. For single faults, we find that dynamic slicing was eight percentage points more effective than the best performing statistical debugging formula; for 66% of the bugs, dynamic slicing finds the fault earlier than the best performing statistical debugging formula. In our evaluation, dynamic slicing is more effective for programs with single fault, but statistical debugging performs better on multiple faults. Best results, however, are obtained by a hybrid approach: If programmers first examine at most the top five most suspicious locations from statistical debugging, and then switch to dynamic slices, on average, they will need to examine 15% (30 lines) of the code. These findings hold for 18 most effective statistical debugging formulas and our results are independent of the number of faults (i.e. single or multiple faults) and error type (i.e. artificial or real errors).

show abstract

An evaluation of pure spectrum‐based fault localization techniques for large‐scale software systems

Cited by 31 publications

References 80 publications

A Survey of Challenges in Spectrum-Based Software Fault Localization

A Survey of Challenges in Spectrum-Based Software Fault Localization

The improved dynamic slicing for spectrum-based fault localization

Locating Faults with Program Slicing: An Empirical Analysis

Contact Info

Product

Resources

About