Challenges in leveraging existing human performance data for quantifying the IDHEAS HRA method

Abstract

This article documents an exploratory study for collecting and using human performance data to inform human error probability (HEP) estimates for a new human reliability analysis (HRA) method, the IntegrateD Human Event Analysis System (IDHEAS). The method was based on cognitive models and mechanisms underlying human behaviour and employs a framework of 14 crew failure modes (CFMs) to represent human failures typical for human performance in nuclear power plant (NPP) internal, at-power events [1]. A decision tree (DT) was constructed for each CFM to assess the probability of the CFM occurring in different contexts. Data needs for IDHEAS quantification are discussed. Then, the data collection framework and process is described and how the collected data were used to inform HEP estimation is illustrated with two examples. Next, five major technical challenges are identified for leveraging human performance data for IDHEAS quantification. These challenges reflect the data needs specific to IDHEAS. More importantly, they also represent the general issues with current human performance data and can provide insight for a path forward to support HRA data collection, use, and exchange for HRA method development, implementation, and validation.

Virtual Modeling of User Populations and Formative Design Parameters

“…Further, surveys and interviews rarely provide measures of performance that can be directly integrated into existing models. Instead, participants usually provide ordinal rankings or qualitative descriptions of their performance which must be converted manually into a quantitative format [19]. Little exact guidance exists on this practice, making researcher experience in the practice critical [20].…”

“…Further, these studies can be controlled such that the observed performance is minimally influenced by confounding factors. The drawback is that these studies can be time consuming and expensive to perform properly [19]. Designers report many difficulties engaging with end-user populations.…”

Challenge to Collect Empirical Data for Human Reliability Analysis—Illustrated by the Difficulties in Collecting Empirical Data on the Performance-Shaping Factor Complexity

“…As it stands, different studies have to be performed based on the different methods to test the different error rates, for the different subcomponents the different methods include. Also Liao et al [6] discuss the 011013-4 / Vol. 6, MARCH 2020…”

“…A particular challenge in analyzing complexity in event reports is that the event reports are usually not very detailed and it is very difficult to understand which subcomponents of complexity are involved. Boring et al [3] and Liao et al [6] also say that the events occur so infrequently that they do not give much data for HRA. Also, another challenge in using event reports to collect HRA data for complexity is that when large accidents happen, such as, for example, the three mile island incident, ATHEANA [10] claims that these accidents are particularly complex.…”

“…The lack of empirical data has often been described as the largest issue in HRA, for example, by, [1,[3][4][5][6][7]. However, the invention and development of HRA methods seemed more to be based on resolving a practical need for estimating human reliability in risk analysis rather than being developed based on empirical research.…”