Constructing a Safety Case for Automatically Generated Code from Formal Program Verification Information

Abstract

Abstract. Formal methods can in principle provide the highest levels of assurance of code safety by providing formal proofs as explicit evidence for the assurance claims. However, the proofs are often complex and difficult to relate to the code, in particular if it has been generated automatically. They may also be based on assumptions and reasoning principles that are not justified. This causes concerns about the trustworthiness of the proofs and thus the assurance claims. Here we present an approach to systematically construct safety cases from information collected during a formal verification of the code, in particular from the construction of the logical annotations necessary for a formal, Hoare-style safety certification. Our approach combines a generic argument that is instantiated with respect to the certified safety property (i.e., safety claims) with a detailed, program-specific argument that can be derived systematically because its structure directly follows the course the annotation construction takes through the code. The resulting safety cases make explicit the formal and informal reasoning principles, and reveal the top-level assumptions and external dependencies that must be taken into account. However, the evidence still comes from the formal safety proofs. Our approach is independent of the given safety property and program, and consequently also independent of the underlying code generator. Here, we illustrate it for the AutoFilter system developed at NASA Ames.

Deriving Safety Cases for Hierarchical Structure in Model-Based Development

“…These subgoals are delayed here as well, to keep the safety case compact. Their expanded structure again follows the lines of our previous work [5], and uses the argumentation shown in Fig. 5 (Tier III) of the safety case there with small modifications; in particular, the notion of safety condition needs to be replaced by that of safety requirement.…”

“…This is a straightforward modification of our previous work on programs without hierarchical system structure (see Fig. 3 Tier I: Explaining the Safety Notion in [5]). Here, we thus focus on the lower part of the safety case that explains that, and how, the generated source code Nav.cpp satisfies the given safety requirements by providing formal proofs as evidence (see Fig.…”

“…This is predicated on the assumptions that the applied Hoare-calculus is sound, and that the VCG is implemented correctly, which need to be justified elsewhere. Since the rest of the safety case is constructed as described in our previous work [5], we do not expand it here any further.…”

“…However, since code generators are typically not qualified, there is no guarantee that their output is correct or even safe, and additional evidence of its safety is required. In previous work [5], we have thus constructed safety cases [19] from information collected during a formal verification of the generated code. We also have constructed safety cases that correspond to the formal proofs found by automated theorem provers of the verification conditions, and reveal the underlying proof argumentation structure and top-level assumptions [6].…”

Generating customized verifiers for automatically generated code

“…For example, in the schema mtrans int shown in Figure 8, the variable A is computed as the transpose of T , so that its frame depends on T 's frame. 5 The mtrans int schema uses a syntactic variant, where the additional (sixth) argument [T] indicates that T is a dependent hot variable for A. Inference will thus proceed past this definition for A and restart, looking for a definition for the new hot variable T .…”