Modeling software design diversity

Abstract

This is the unspecified version of the paper.This version of the publication may differ from the final published version. Permanent repository link: AbstractDesign diversity has been used for many years now as a means of achieving a degree of fault tolerance in software-based systems. Whilst there is clear evidence that the approach can be expected to deliver some increase in reliability compared with a single version, there is not agreement about the extent of this. More importantly, it remains difficult to evaluate exactly how reliable a particular diverse faulttolerant system is. This difficulty arises because assumptions of independence of failures between different versions have been shown not to be tenable: assessment of the actual level of dependence present is therefore needed, and this is hard. In this tutorial we survey the modelling issues here, with an emphasis upon the impact these have upon the problem of assessing the reliability of fault tolerant systems. The intended audience is one of designers, assessors and project managers with only a basic knowledge of probabilities, as well as reliability experts without detailed knowledge of software, who seek an introduction to the probabilistic issues in decisions about design diversity.

Effects of incorrect computer-aided detection (CAD) output on human decision-making in mammography

“…"Reliability modelling" in engineering is concerned with predicting probabilities of future failures 1 (or failure-free behaviour) of man-made systems. It most typically involves building a "model" which gives a mathematical specification of how failures of its components may cause the failure of the system (specifically, which combinations of component failures would lead to system failure) (1). These models bring insight about which components are most critical for the dependability of the whole system, about which component it is best to spend effort to improve, and sometimes allow one to estimate the dependability measures for the whole system, like the probability of the system behaving improperly, from parameters like the known reliability levels of the individual components.…”

“…As is common practice in the evaluation of medical products, the HTA trial compared the effectiveness of human film-readers' behaviour with and without the 1 We will use "failure" to mean any case in which something (a machine or a human or a set of cooperating humans and machines) produces an incorrect output. 2 Henceforth in this paper we will always use the term "system" to refer to the whole system which includes a human and a machine component.…”

Conservative claims about the probability of perfection of software-based systems

“…Figure 2 is intended to depict this infimum property of this set of distributions. The infimum of the resulting set of Bayesian posterior perfection probabilities -each computed from one member of this set of priors -is easily show to be the RHS of equation (4).…”

“…So if, in a 1-out-of-2 protection system (1oo2 system), channel A fails on a particular demand, there may be a good chance that channel B will not fail. There is evidence from some industrial applications that this kind of design diversity has been successful [4]. For example, the safetycritical flight control systems of Airbus fleets have experienced massive operational exposure [5] with apparently no critical failure.…”

Uncertainty Explicit Assessment of Off-the-Shelf Software: Selection of an Optimal Diverse Pair

“…The components that form the best pair may not necessarily be the ones which are the best individually. For further details on the subtleties of this problem the interested reader is referred to a recent survey [16].…”