An Industrial Survey of Safety Evidence Change Impact Analysis Practice

Abstract

Abstract-Context. In many application domains, critical systems must comply with safety standards. This involves gathering safety evidence in the form of artefacts such as safety analyses, system specifications, and testing results. These artefacts can evolve during a system's lifecycle, creating a need for change impact analysis to guarantee that system safety and compliance are not jeopardised. Objective. We aim to provide new insights into how safety evidence change impact analysis is addressed in practice. The knowledge about this activity is limited despite the extensive research that has been conducted on change impact analysis and on safety evidence management. Method. We conducted an industrial survey on the circumstances under which safety evidence change impact analysis is addressed, the tool support used, and the challenges faced. Results. We obtained 97 valid responses representing 16 application domains, 28 countries, and 47 safety standards. The respondents had most often performed safety evidence change impact analysis during system development, from system specifications, and fully manually. No commercial change impact analysis tool was reported as used for all artefact types and insufficient tool support was the most frequent challenge. Conclusion. The results suggest that the different artefact types used as safety evidence co-evolve. In addition, the evolution of safety cases should probably be better managed, the level of automation in safety evidence change impact analysis is low, and the state of the practice can benefit from over 20 improvement areas.

An analysis of safety evidence management with the Structured Assurance Case Metamodel

“…To the best of our knowledge, no publication up to now has studied in depth how safety evidence management can be addressed with all the classes and associations of SACM. Regarding prior work on safety evidence management, recent publications have presented large studies on the state of the art [5] and on the state of the practice [6,7]. It is also easy to find deliverables in research projects that have reviewed the literature and industrial practices (e.g., [26]).…”

“…In addition, no standard definition of safety evidence exists and we have adopted a definition [5] as basis for the analysis. Nonetheless, such a definition is based on a large-scale systematic literature review [5] and has been later validated in surveys with practitioners [6,7]. This makes us confident in its validity.…”

Software Engineers' Information Seeking Behavior in Change Impact Analysis - An Interview Study

“…Most CIA work in industry is manual [16], although the importance of improved CIA tools has been highlighted in research for a long time [17]. Also, two recent reviews of scientific literature shows that most research on CIA is limited to impact on source code [15], [18].…”

Practitioners’ Perspectives on Change Impact Analysis for Safety-Critical Software – A Preliminary Analysis

“…IEC 61508 states that "if at any phase of the software safety lifecycle, a modification is required pertaining to an earlier lifecycle phase, then an impact analysis shall determine (1) which software modules are impacted, and (2) which earlier safety lifecycle activities shall be repeated." CIA is often a difficult task in practice due to the size and complexity of safetycritical systems [2,7]. Inadequate CIA has further been among the causes of accidents and near-accidents in the past [10].…”

“…the alignment of requirements with verification and validation [1] and on traceability [13]. Regarding the surveys, Nair et al [12] studied safety evidence management practices, including certain aspects related to change management, and de la Vara et al [7] conducted a survey on safety evidence CIA to explore the circumstances under which it is performed, the tool support used, and the challenges faced.…”