NFTAPE: a framework for assessing dependability in distributed systems with lightweight fault injectors

Stott, D.T.; Floering, B.; Burke, David; Kalbarczpk, Z.; Iyer, Ravishankar K.

doi:10.1109/ipds.2000.839467

Cited by 138 publications

(69 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…So called lightweight fault injector components are defined to perform the actual injection, monitor and trigger components can similarly be provided by the user [Stott et al, 2000]. NFTAPE has been used to study error sensitivity of Linux [Gu et al, 2004].…”

Section: Fault Injectionmentioning

confidence: 99%

See 1 more Smart Citation

Robustness Evaluation of Operating Systems

Johansson

2008

Information Assurance

View full text Add to dashboard Cite

SummaryThe premise behind this thesis is the observation that Operating Systems (OS), being the foundation behind operations of computing systems, are complex entities and also subject to failures. Consequently, when they do fail, the impact is the loss of system service and the applications running thereon. While a multitude of sources for OS failures exist, device drivers are often identified as a prominent cause behind failures.In order to characterize the impact of driver failures, at both the OS and application levels, this thesis develops a framework for error propagation-based robustness profiling for an OS. The framework is first developed conceptually and then experimentally validated on a real OS, namely Windows CE .Net. The choice of Windows CE is driven by its representativeness for a multitude of OS's, as well as the ability to customize the OS components for particular needs.For experimental validation, fault injection is a prominent technique that can be used to simulate faults (or errors) in the system by inserting synthetic ones and study their effect. Three key questions with such a technique are where, what and when to inject faults. This thesis shows how injecting errors at the interface between drivers and the OS can be very effective in evaluating the effects driver faults can have.To quantify the OS's robustness, this thesis defines a series of error propagation measures, specifically tailored for device drivers. These measures allow for quantifying and comparing both individual services and device drivers on their susceptibility and diffusing abilities.This thesis compares three contemporary error models on their suitability for robustness evaluation. The classical bit-flip model is found to identify a higher number of severe failures in the system. It also identifies failures for more services than both other models, data type and fuzzing. However, its main drawback is that it requires substantially more injections than the other two. Fuzzing, even though not giving rise to as many failures is able to find new additional services with severe failures.A careful study of the injections performed with the bit-flip model shows that only a few bits are generally useful for identifying new services with robustness weaknesses. Consequently, a new composite model is proposed, combining the most effective bits of the bit-flip model with the fuzzing model's ability to identify new services, giving rise to new model without loss of important information and at the same time incurring a moderate number of injections.To answer the question of when to inject an error this thesis proposes a novel model of a driver's usage profile, focusing on high-level operations being carried out. It guides the injection of errors to instances when the driver is carrying out specific operations. Results from extensive fault injection experiments show that more service vulnerabilities can be discovered. Furthermore, a priori profiling of the drivers can show how effective the proposed approach will be. vi

show abstract

Section: Fault Injectionmentioning

confidence: 99%

“…In this work we have focused on a transient error model as we believe this to more closely represent behavior by the drivers not found through standard testing routines. Multiple other fault injection tools allow for injection of both transient, intermittent and permanent faults, e.g., [Han et al, 1995;Stott et al, 2000].…”

Section: Error Triggermentioning

confidence: 99%

Robustness Evaluation of Operating Systems

Johansson

2008

Information Assurance

View full text Add to dashboard Cite

show abstract

“…In particular fault injection aims at (1) exposing deficiencies of fault tolerance mechanisms (i.e., fault removal), e.g., [3], and (2) evaluating coverage of fault tolerance mechanisms (i.e., fault forecasting, e.g., [2]. Number of tools were proposed to support fault injection analysis and evaluation of systems, e.g., FERRARI [14], FIAT [5], and NFTAPE [22]. This section presents an example of applying fault/error injection in assessing fault tolerance mechanisms of software implemented fault tolerance environment for spaceborne applications.…”

Section: Fault/error Injection Characterization Of the Sift Environmementioning

confidence: 99%

Measurement-Based Analysis of System Dependability Using Fault Injection and Field Failure Data

Iyer

Kalbarczyk

2002

Performance Evaluation of Complex Systems: Techniques and Tools

View full text Add to dashboard Cite

Abstract. The discussion in this paper focuses on the issues involved in analyzing the availability of networked systems using fault injection and the failure data collected by the logging mechanisms built into the system. In particular we address: (1) analysis in the prototype phase using physical fault injection to an actual system. We use example of fault injection-based evaluation of a software-implemented fault tolerance (SIFT) environment (built around a set of self-checking processes called ARMORS) that provides error detection and recovery services to spaceborne scientific applications and (2) measurement-based analysis of systems in the field. We use example of LAN of Windows NT based computers to present methods for collecting and analyzing failure data to characterize network system dependability. Both, fault injection and failure data analysis enable us to study naturally occurring errors and to provide feedback to system designers on potential availability bottlenecks. For example, the study of failures in a network of Windows NT machines reveals that most of the problems that lead to reboots are software related and that though the average availability evaluates to over 99%, a typical machine, on average, provides acceptable service only about 92% of the time.

show abstract

“…The use of fault injection to test distributed systems [19,7,8,6] and leveraging virtualization for fault injection [2,3,18,16,9,12] have been presented in previous works. However, unique features of our work include: A) a comprehensive presentation of all aspects of a practical testing infrastructure, and B) ad etailed case study of the use of the infrastructure to validate a system with multiple advanced fault tolerance features.…”

Section: Introductionmentioning

confidence: 99%

Using Virtualization to Validate Fault-Tolerant Distributed Systems

Hsu

Gallagher

et al. 2010

Informatics

View full text Add to dashboard Cite

Asynchronous events and complexs ystem state distributed across independent nodes makee xposure and diagnosis of flaws in distributed systems a challenge. The difficulties are exacerbated when the goal is to validate fault tolerance mechanisms that are activated only by the occurrence of errors, which are, by nature, rare. Va lidation of fault tolerance mechanisms is often done by injecting faults that emulate the actual faults and ''stress''t he functionality of the resilience mechanisms. Va lidation campaigns lasting days and involving thousands of fault injections are often necessary.W ep resent an infrastructure that combines virtualization and software-implemented fault injection to automate validation campaigns and support the analysis of the behavior of a distributed system under test. Virtualization enables: 1) aflexible fault injector capable of emulating a wide variety of faults, and 2) am echanism for autonomously recovering faulty nodes so that the campaign can continue running on a target system that is fully functional. As ac ase study we use this infrastructure to validate a Byzantine-fault-tolerant cluster manager.O ver 1280 hours of fault injections yielded the exposure of 11 unique flaws in the cluster manager.

show abstract

NFTAPE: a framework for assessing dependability in distributed systems with lightweight fault injectors

Cited by 138 publications

References 16 publications

Robustness Evaluation of Operating Systems

Robustness Evaluation of Operating Systems

Measurement-Based Analysis of System Dependability Using Fault Injection and Field Failure Data

Using Virtualization to Validate Fault-Tolerant Distributed Systems

Contact Info

Product

Resources

About