Assessment of diagnostic test for automated bug localization

Abstract

Statistical simulation based design error debug approaches strongly rely on quality of the diagnostic test. At the same time there exists no dedicated technique to perform its quality assessment and engineers are forced to rely on subjective figures such as verification test quality metrics or just the size of the diagnostic test. This paper has proposed two new approaches for assessing diagnostic capability of diagnostic tests for automated bug localization. The first approach relies on probabilistic simulation of diagnostic experiments. The second assessment method is based on calculating Hamming distances of the individual sub-tests in the diagnostic test set. The methods are computationally cheap and they provide for a measure of confidence in the localization results and allow estimating impact of the diagnostic test enhancement. The approach is implemented as a part of an open-source hardware design and debugging framework zamiaCAD. Experimental results with an industrial processor design and a set of documented bugs demonstrate feasibility and effectiveness of the proposed approach.