Evaluation and Analysis of Spectrum-Based Fault Localization with Modified Similarity Coefficients for Software Debugging

Abstract

During the process of fault localization, the spectrum-based techniques are frequently used and widely studied since they can automatically and effectively localize the faults of software and be implemented easily. So far most of spectrum-based fault localization techniques have relied heavily on the use of similarity coefficients. However, we noticed that existing similarity coefficients for fault localization may lack measure(s) to properly reflect the relationship between failing and passing test cases. It also has to note that the failing test cases usually are expected to provide more information to the similarity coefficients than the passing test cases. In order to evaluate the importance of failing and passing test cases in the similarity coefficients, a number of modified similarity coefficients in fault localization are presented and discussed. The modified similarity coefficients which are assigned the weights of the failing and/or passing test cases will be studied and analyzed with the multi-coverage-combined techniques. Five open source programs and 75 faulty versions in total from Siemens suite, which have been widely used for software testing and comparison of fault localization techniques, were selected as experiment subjects. Detailed analysis of the results shows that assigning the weights of failing and passing test cases to the similarity coefficients would be able to localize the faults more effectively and accurately.

Incremental spectrum cloning algorithm for optimization of spectrum-based fault localization

“…Despite its simplicity in approach, the proposed incremental spectrum cloning algorithm has significantly improved the cost effectiveness of the SBFL because less lines of code need to be inspected in order to locate the faulty code. This is consistent with results from a related study [11] which attempted to optimize the performance of SBFL ranking metrics by assigning higher weighting to fail test cases. The proposed incremental spectrum cloning algorithm not only optimized the performance of SBFL ranking metrics, but also provided a viable solution to the Patrick Daniel et al…”

A Framework for Improving Fault Localization Effectiveness Based on Fuzzy Expert System

“…• EXAM: EXAM [6], [10], one of the most frequently used metrics, indicates the percentage of statements in a program that have to be examined until the first faulty statement is reached. The lower the value of this percentage metric is, the more effective the fault localization is.…”

Methods for Enhancing Software Fault Localization

“…Also, the authors did not evaluate the improved Tarantula using well-known evaluation metrics. The authors in [171] modified three well-known SBFL formulas based on the idea that some failed test cases may provide more testing information than other failed test cases. Therefore, for the three used formulas, different weights for failed test cases were assigned and then applied with multi-coverage spectra.…”