Safety-informed design: Using subgraph analysis to elicit hazardous emergent failure behavior in complex systems

Abstract

Identifying failure paths and potentially hazardous scenarios resulting from component faults and interactions is a challenge in the early design process. The inherent complexity present in large engineered systems leads to nonobvious emergent behavior, which may result in unforeseen hazards. Current hazard analysis techniques focus on single hazards (fault trees), single faults (event trees), or lists of known hazards in the domain (hazard identification). Early in the design of a complex system, engineers may represent their system as a functional model. A function failure reasoning tool can then exhaustively simulate qualitative failure scenarios. Some scenarios can be identified as hazardous by hazard rules specified by the engineer, but the goal is to identify scenarios representing unknown hazards. The incidences of specific subgraphs in graph representations of known hazardous scenarios are used to train a classifier to distinguish hazard from nonhazard. The algorithm identifies the scenario most likely to be hazardous, and presents it to the engineer. After viewing the scenario and judging its safety, the engineer may have insight to produce additional hazard rules. The collaborative process of strategic presentation of scenarios by the computer and human judgment will identify previously unknown hazards. The feasibility of this methodology has been tested on a relatively simple functional model of an electrical power system with positive results. Related work applying function failure reasoning to a team of robotic rovers will provide data from a more complex system.

A Better Understanding of Safety of Engineered Systems

“…There have been numerous highprofile, expensive, and occasionally tragic accidents, frequently preceded by a number of seemingly unrelated, but interconnected, incidents. The Deep Water Horizon disaster's incalculable economic, environmental, and human costs (Summerhayes, 2011), the recent grounding of the Boeing 787 line, which is estimated to have cost $5 billion, and the accident involving the space shuttle Columbia (McIntire et al, 2016) are all examples of the unacceptably high cost of addressing complex failures and safety too late. Unexpected or unforeseen interactions between system components lead to system accidents.…”