Evaluating Distortion in Fault Injection Experiments

Kouwe, Erik van der; Giuffrida, Cristiano; Tanenbaum, Andrew S.

doi:10.1109/hase.2014.13

Cited by 9 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, the combined coverage reached is quite low, especially on Linux. While it would be desirable to reach higher coverage [22], it is hard to do so because we can only use features supported by both operating systems in our workloads. It may be possible to reach higher coverage in systems that are more similar, but then there is still the issue that much of the hardware support is not used, especially when run in an emulator.…”

Section: Resultsmentioning

confidence: 99%

“…According to [22], the most representative way to select faults is to make the chance of selecting a certain location or fault type proportional to the number of fault candidates. This means that larger components have a proportionally larger chance of being selected (consistent with [1]) and fault types for which there are many opportunities to introduce are injected more often.…”

Section: B Fault Selectionmentioning

confidence: 99%

“…A fault candidate is a pair of a code location and a fault type that can be injected at that location [22]. Fault candidates can be identified at three different levels: source code, intermediate code, and binary code.…”

Section: A Fault Injectionmentioning

confidence: 99%

See 2 more Smart Citations

A Methodology to Efficiently Compare Operating System Stability

Kouwe¹,

Giuffriday²,

Ghituletez³

et al. 2015

2015 IEEE 16th International Symposium on High Assurance Systems Engineering

Self Cite

View full text Add to dashboard Cite

Despite decades of advances in software engineering, operating systems (OSes) are still plagued by crashes due to software faults, calling for techniques to improve OS stability when faults occur. Evaluating such techniques requires a way to compare the stability of different OSes that is both representative of real faults and scales to the large code bases of modern OSes and a large (and statistically sound) number of experiments.In this paper, we propose a widely applicable methodology meeting all such requirements. Our methodology relies on a novel fault injection strategy based on a combination of static and runtime instrumentation, which yields representative software faults while drastically reducing the instrumentation time and thus greatly enhancing scalability. To guarantee unbiased and comparable results, finally, our methodology relies on the use of pre-and posttests to isolate the direct impact of faults from the stability of the OS itself. We demonstrate our methodology by comparing the stability of Linux and MINIX 3, saving a total of 115 computerdays for the 12,000 Linux fault injection runs compared to the traditional approach of re-instrumenting for every run.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: B Fault Selectionmentioning

confidence: 99%

See 1 more Smart Citation

A Methodology to Efficiently Compare Operating System Stability

Kouwe¹,

Giuffriday²,

Ghituletez³

et al. 2015

2015 IEEE 16th International Symposium on High Assurance Systems Engineering

Self Cite

View full text Add to dashboard Cite

show abstract

“…This effort is driven using the regression tests that accompany the targeted MPI implementations. This is a common method [32] of achieving high coverage without resorting to random API testing. MPI regression tests use either the MPI_ERRORS_RETURN or the MPI_ERRORS_ARE_FATAL modes.…”

Section: Reproducing Error Propagation Bugsmentioning

confidence: 99%

Detecting and reproducing error-code propagation bugs in MPI implementations

DeFreez

Antara

Laguna

et al. 2020

Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

View full text Add to dashboard Cite

We present an approach to automatically detect and reproduce error code propagation bugs in MPI implementations. Specifically, we combine static analysis and program repair for bug detection, and apply fault injection to reproduce error propagation bugs found in MPI libraries written in C. We demonstrate our approach on the MPICH library, one of the most popular implementations of MPI, and the MPICHbased implementation MVAPICH, uncovering 447 previously unknown bugs. We discovered that 31 of these bugs result in program crashes, and 60% of the MPICH test suite is susceptible to crashing due to failures to propagate error codes. Moreover, 95 bugs produce undesirable behavior that has been confirmed dynamically, causing tests to fail, hanging processes, or simply dropping error codes before reaching user applications. • Software and its engineering → Automated static analysis; Dynamic analysis; Error handling and recovery; • Computing methodologies → Parallel programming languages.

show abstract

“…Please note that this is an extended version of the earlier conference paper (van der Kouwe et al 2014), which shares these contributions. Among others, this version has a more complete empirical evaluation and discusses the methodology and its limitations in more depth.…”

mentioning

confidence: 93%

Finding fault with fault injection: an empirical exploration of distortion in fault injection experiments

Kouwe¹,

Giuffrida²,

Tanenbaum³

2014

Software Qual J

Self Cite

View full text Add to dashboard Cite

It has become well established that software will never become bug free, which has spurred research in mechanisms to contain faults and recover from them. Since such mechanisms deal with faults, fault injection is necessary to evaluate their effectiveness. However, little thought has been put into the question whether fault injection experiments faithfully represent the fault model designed by the user. Correspondence with the fault model is crucial to be able to draw strong and general conclusions from experimental results. The aim of this paper is twofold: to make a case for carefully evaluating whether activated faults match the fault model and to gain a better understanding of which parameters affect the deviation of the activated faults from the fault model. To achieve the latter, we instrumented a number of programs with our LLVM-based fault injection framework. We investigated the biases introduced by limited coverage, parts of the program executed more often than others and the nature of the workload. We evaluated the key factors that cause activated faults to deviate from the model and from these results provide recommendations on how to reduce such deviations.

show abstract

Evaluating Distortion in Fault Injection Experiments

Cited by 9 publications

References 23 publications

A Methodology to Efficiently Compare Operating System Stability

A Methodology to Efficiently Compare Operating System Stability

Detecting and reproducing error-code propagation bugs in MPI implementations

Finding fault with fault injection: an empirical exploration of distortion in fault injection experiments

Contact Info

Product

Resources

About