Experimental analysis of the errors induced into Linux by three fault injection techniques

Jarboui, Tahar; Arlat, Jean; Crouzet, Yves; Kanoun, Karama

doi:10.1109/dsn.2002.1028917

Cited by 18 publications

(17 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is in line with multiple previous studies, e.g., [Durães and Madeira, 2003], [Gu et al, 2003] and [Jarboui et al, 2002a]. Experiments in the Class NF category are either masked by the system, for instance parameters not used in this context or overwritten; or handled by built-in error detection/correction mechanisms checking incoming parameter values for correctness.…”

Section: Number Of Failuressupporting

confidence: 75%

“…The work reported in [Albinet et al, 2004;Durães and Madeira, 2003;Gu et al, 2004;Jarboui et al, 2002a] explored the use of various error models and injection techniques for OS robustness evaluation and benchmarking. In [Jarboui et al, 2002a], for instance, error models similar to ours are used, but are injected at different levels within the Linux kernel.…”

Section: Other Contemporary Software Error Modelsmentioning

confidence: 99%

“…In [Jarboui et al, 2002a], for instance, error models similar to ours are used, but are injected at different levels within the Linux kernel. Durães et.…”

Section: Other Contemporary Software Error Modelsmentioning

confidence: 99%

See 2 more Smart Citations

Robustness Evaluation of Operating Systems

Johansson

2008

Information Assurance

View full text Add to dashboard Cite

SummaryThe premise behind this thesis is the observation that Operating Systems (OS), being the foundation behind operations of computing systems, are complex entities and also subject to failures. Consequently, when they do fail, the impact is the loss of system service and the applications running thereon. While a multitude of sources for OS failures exist, device drivers are often identified as a prominent cause behind failures.In order to characterize the impact of driver failures, at both the OS and application levels, this thesis develops a framework for error propagation-based robustness profiling for an OS. The framework is first developed conceptually and then experimentally validated on a real OS, namely Windows CE .Net. The choice of Windows CE is driven by its representativeness for a multitude of OS's, as well as the ability to customize the OS components for particular needs.For experimental validation, fault injection is a prominent technique that can be used to simulate faults (or errors) in the system by inserting synthetic ones and study their effect. Three key questions with such a technique are where, what and when to inject faults. This thesis shows how injecting errors at the interface between drivers and the OS can be very effective in evaluating the effects driver faults can have.To quantify the OS's robustness, this thesis defines a series of error propagation measures, specifically tailored for device drivers. These measures allow for quantifying and comparing both individual services and device drivers on their susceptibility and diffusing abilities.This thesis compares three contemporary error models on their suitability for robustness evaluation. The classical bit-flip model is found to identify a higher number of severe failures in the system. It also identifies failures for more services than both other models, data type and fuzzing. However, its main drawback is that it requires substantially more injections than the other two. Fuzzing, even though not giving rise to as many failures is able to find new additional services with severe failures.A careful study of the injections performed with the bit-flip model shows that only a few bits are generally useful for identifying new services with robustness weaknesses. Consequently, a new composite model is proposed, combining the most effective bits of the bit-flip model with the fuzzing model's ability to identify new services, giving rise to new model without loss of important information and at the same time incurring a moderate number of injections.To answer the question of when to inject an error this thesis proposes a novel model of a driver's usage profile, focusing on high-level operations being carried out. It guides the injection of errors to instances when the driver is carrying out specific operations. Results from extensive fault injection experiments show that more service vulnerabilities can be discovered. Furthermore, a priori profiling of the drivers can show how effective the proposed approach will be. vi

show abstract

Section: Number Of Failuressupporting

confidence: 75%

Section: Other Contemporary Software Error Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Robustness Evaluation of Operating Systems

Johansson

2008

Information Assurance

View full text Add to dashboard Cite

show abstract

“…Fault injection techniques are widely used for software and system testing [13,20,21,22,24], ranging from testing the reliability of device drivers to testing operating systems, embedded systems and real-time systems [3,7,14,15,18,23,27].…”

Section: Fault Injection Techniquesmentioning

confidence: 99%

Automatic fault injection for driver robustness testing

Cong

Yang

et al. 2015

Proceedings of the 2015 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Robustness testing is a crucial stage in the device driver development cycle. To accelerate driver robustness testing, effective fault scenarios need to be generated and injected without requiring much time and human effort. In this paper, we present a practical approach to automatic runtime generation and injection of fault scenarios for driver robustness testing. We identify target functions that can fail from runtime execution traces, generate effective fault scenarios on these target functions using a bounded trace-based iterative strategy, and inject the generated fault scenarios at runtime to test driver robustness using a permutation-based injection mechanism. We have evaluated our approach on 12 Linux device drivers and found 28 severe bugs. All these bugs have been further validated via manual fault injection. The results demonstrate that our approach is useful and efficient in generating fault scenarios for driver robustness testing with little manual effort.

show abstract

“…Fault injection techniques are widely used for software and system testing [29,54,55,56,61], ranging from testing the reliability of device drivers to testing operating systems, embedded systems and real-time systems [5,16,35,36,46,58,67].…”

Section: Fault Injection Techniquesmentioning

confidence: 99%

Post-silicon Functional Validation with Virtual Prototypes

Cong¹

2000

View full text Add to dashboard Cite

Post-silicon validation has become a critical stage in the system-on-chip (SoC) development cycle, driven by increasing design complexity, higher level of integration and decreasing time-to-market. According to recent reports, post-silicon validation effort comprises more than 50% of the overall development effort of an 65nmSoC. Though post-silicon validation covers many aspects ranging from electronic properties of hardware to performance and power consumption of whole systems, a central task remains validating functional correctness of both hardware and its integration with software. There are several key challenges to achieving accelerated and low-cost post-silicon functional validation. First, there is only limited silicon observability and controllability; second, there is no good test coverage estimation over a silicon device; third, it is difficult to generate good post-silicon tests before a silicon device is available; fourth, there is no effective software robustness testing approaches to ensure the quality of hardware/software integration.We propose a systematic approach to accelerating post-silicon functional validation with virtual prototypes. Post-silicon test coverage is estimated in the presilicon stage by evaluating the test cases on the virtual prototypes. Such analysis is first conducted on the initial test suite assembled by the user and subsequently on the expanded test suite which includes test cases that are automatically generated.Based on the coverage statistics of the initial test suite on the virtual prototypes, ii test cases are automatically generated to improve the test coverage. In the postsilicon stage, our approach supports coverage evaluation of test cases on silicon devices to ensure fidelity of early coverage evaluation. The generated test cases are issued to silicon devices to detect inconsistencies between virtual prototypes and silicon devices using conformance checking. We further extend the test case generation framework to generate and inject fault scenario with virtual prototypes for driver robustness testing. Besides virtual prototype-based fault injection, an automatic driver fault injection approach is developed to support runtime fault generation and injection for driver robustness testing. Since virtual prototype enables early driver development, our automatic driver fault injection approach can be applied to driver testing in both pre-silicon and post-silicon stages.For preliminary evaluation, we have applied our coverage evaluation and test generation to several network adapters and their virtual prototypes. We have conducted coverage analysis for a suite of common tests on both the virtual prototypes and silicon devices. The results show that our approach can estimate the test coverage with high fidelity. Based on the coverage estimation, we have employed our automatic test generation approach to generate additional tests. When the generated test cases were issued to both virtual prototypes and silicon devices, we observed significant coverage improvemen...

show abstract

Experimental analysis of the errors induced into Linux by three fault injection techniques

Cited by 18 publications

References 11 publications

Robustness Evaluation of Operating Systems

Robustness Evaluation of Operating Systems

Automatic fault injection for driver robustness testing

Post-silicon Functional Validation with Virtual Prototypes

Contact Info

Product

Resources

About