A Deliberative Method for Ranking Risks (I): Overview and Test Bed Development

Abstract

Risk ranking offers a potentially powerful means for gathering public input to help set risk-management priorities. In most rankings conducted to date, the categories and attributes used to describe the risks have varied widely, the materials and procedures have not been designed to facilitate comparisons among risks on all important attributes, and the validity and reproducibility of the resulting rankings have not been assessed. To address these needs, a risk-ranking method was developed in which risk experts define and categorize the risks to be ranked, identify the relevant risk attributes, and characterize the risks in a set of standardized risk summary sheets, which are then used by lay or other groups in structured ranking exercises. To evaluate this method, a test bed involving 22 health and safety risks in a fictitious middle school was created. This article provides an overview of the risk-ranking method and describes the challenges faced in designing the middle school test bed. A companion article in this issue reports on the validity of the ranking procedures and the level of agreement among risk managers regarding ranking of risks and attributes.

Health risk communication using comparative risk analyses

“…Consequently, a number of visual aids continue to be developed and tested to help people better understand seemingly complex risk information and put risk magnitudes into proper perspective. These include risk ladders, pie charts, dots, and community risk scales (Hammitt, 1990;Gutteling and Wiegman, 1996;Sandman et al, 1994;Weinstein et al, 1996;Calman and Royston, 1997;Siegrist, 1997;Florig et al, 2001;Morgan et al, 2001). Verbal analogies, such as comparing a risk to its equivalent in time, distance, population, or games of chance, have also been used to present risk information (Weinstein et al, 1996).…”

“…For this latter effort in particular (i.e., putting risk data into appropriate context), various approaches have been developed or recommended to help convey risk magnitudes, including the use of risk comparisons, risk rankings, verbal analogies, and a host of other visual aids (e.g., risk ladders, pie charts, dots, community scales) (Hammitt, 1990;Gutteling and Wiegman, 1996;Sandman et al, 1994;Weinstein et al, 1996; Calman and Royston, 1997; Siegrist, 1997;Florig et al, 2001. Although there has been some attempt to test or compare empirically these alternative risk communication methods, it remains unclear as to which approach is the most useful or appropriate under different circumstances (Roth et al, 1990;Slovic et al, 1990;Weinstein and Sandman, 1993;Freudenburg and Rursch, 1994;Sandman et al, 1994;Johnson, 2002Johnson, , 2003aJohnson, , 2004.…”

“…A variety of fact sheets, handbooks, and other types of publications are also available that summarize the important aspects and pitfalls of health risk communication (Fessenden-Raden et al, 1987;Sandman, 1987;Covello et al, 1989;Hance et al, 1989Hance et al, , 1990Weinstein et al, 1989;Patton, 1993;Chess and Hance, 1994;Lundgren, 1994). According to these publications, some of the key principles for increasing the effectiveness of risk communication efforts include: (1) engage in a dialogue with the audience, (2) explain risk information, (3) simplify and personalize risk messages, (4) recognize the validity of public concerns, (5) address data uncertainty, and (6) put risk data into appropriate context.For this latter effort in particular (i.e., putting risk data into appropriate context), various approaches have been developed or recommended to help convey risk magnitudes, including the use of risk comparisons, risk rankings, verbal analogies, and a host of other visual aids (e.g., risk ladders, pie charts, dots, community scales) (Hammitt, 1990;Gutteling and Wiegman, 1996;Sandman et al, 1994;Weinstein et al, 1996; Calman and Royston, 1997; Siegrist, 1997;Florig et al, 2001. Although there has been some attempt to test or compare empirically these alternative risk communication methods, it remains unclear as to which approach is the most useful or appropriate under different circumstances (Roth et al, 1990;Slovic et al, 1990;Weinstein and Sandman, 1993;Freudenburg and Rursch, 1994;Sandman et al, 1994;Johnson, 2002Johnson, , 2003aJohnson, , 2004.…”

Deliberative Risk Ranking to Inform Homeland Security Strategic Planning

“…In response, Morgan et al proposed a framework for a risk-ranking method that could engage a wide range of stakeholder participation in a systematic process that used multiple quantitative and qualitative estimates of consequence (Morgan et al 1996). Later papers developed the framework into a systematic process called the Deliberative Method for Ranking Risks (Jenni 1997;Morgan et al 2000;Florig et al 2001;Morgan et al 2001).…”

“…First, some characteristics of risk are not easily measured using quantitative approaches because of either lack of valid measures or lack of reliable data. For example, the psychologic impacts of risks or environmental consequences of risk have been characterized using qualitative methods in past comparative risk analysis studies (Morgan 1999;Florig et al 2001;Morgan et al 2001;Willis et al 2004). Second, for a quantitative approach requires using importance-weights to aggregate across different types of consequences.…”