Use of Technical Expert Panels: Applications to Probabilistic Seismic Hazard Analysis*

Abstract

Probabilistic Seismic Hazard Analysis (PSHA) is a methodology that estimates the likelihood that various levels of earthquake-caused ground motions will be exceeded at a given location in a given future time period. Due to large uncertainties in all of the geosciences data and in their modeling, multiple model interpretations are often possible. This leads to disagreements among the experts, which in the past has led to disagreement on the selection of a ground motion for design at a given site. This paper reports on a project, co-sponsored by the U.S. Nuclear Regulatory Commission, the U.S. Department of Energy, and the Electric Power Research Institute, that was undertaken to review the state-of-the-art and improve on the overall stability of the PSHA process, by providing methodological guidance on how to perform a PSHA. The project reviewed past studies and examined ways to improve on the present state-of-the-art. In analyzing past PSHA studies, the most important conclusion is that differences in PSHA results are commonly due to process rather than technical differences. Thus, the project concentrated heavily on developing process recommendations, especially on the use of multiple experts, and this paper reports on those process recommendations. The problem of facilitating and integrating the judgments of a diverse group of experts is analyzed in detail. The authors believe that the concepts and process principles apply just as well to non-earthquake fields such as volcanic hazard, flood risk, nuclear-plant safety, and climate change.*This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, expressed or implied, or assumes any legal liability or responsibility for any third party's use or the results of such use, of any information, apparatus, product, or process disclosed in this report, or represents that its use by such third party would not infringe privately owned rights. The views expressed in this paper are not necessarily those of the U.S. Nuclear Regulatory Commission.

Use (and abuse) of expert elicitation in support of decision making for public policy

“…The method used the construction of probability trees that allowed experts to make judgments about the relative likelihood that alternative models of possible pharmacokinetic and pharmacodynamic processes correctly describe the biological process that are involved. Budnitz et al (55)(56)(57) have used a set of deliberative processes designed to support a group of experts in developing a "composite probability distribution [that] represents the overall scientific community." The process they developed is very labor intensive and uses experts as evaluators of alternative causal models and their implications rather than as proponents of one or another model.…”

“…The process they developed is very labor intensive and uses experts as evaluators of alternative causal models and their implications rather than as proponents of one or another model. It would be highly desirable to apply procedures such as those developed and demonstrated by Evans et al (53,54) and Budnitz et al (55,56) in assessment processes such as that used by the Intergovernmental Panel on Climate Change (IPCC). However, resource constraints and the limited familiarity that most experts have with decision science, probably makes such an effort infeasible.…”

Survey of sampling-based methods for uncertainty and sensitivity analysis.

“…The distributions D 1 , D 2 , …, D nX are typically defined through an expert review process, [93][94][95][96][97][98][99][100] and their development can constitute a major analysis cost. A possible analysis strategy is to perform an initial exploratory analysis with rather crude definitions for D 1 , D 2 , …, D nX and use sensitivity analysis to identify the most important analysis inputs; then, resources can be concentrated on characterizing the uncertainty in these inputs and a second presentation or decision-aiding analysis can be carried out with these improved uncertainty characterizations.…”

How to interpret expert judgment assessments of 21st century sea-level rise

“…There are many examples of geophysical processes for which the physical mechanisms driving changes are only partly understood (e.g., Cooke 1991;Budnitz et al 1998;Zickfeld et al 2007Zickfeld et al , 2010. As a result, projecting future changes cannot solely rely on deterministic models.…”