The Reliability of Dangerousness Assessments. A Decision Making Exercise

Abstract

One hundred and ninety three raters drawn from six countries and representing different professional groups considered 16 case histories. Raters were asked to indicate what degree of 'dangerousness' they attributed to each individual and what they considered to be the optimal management. The level of agreement between raters concerning the assessment of dangerousness was generally low, the level of 60 per cent being reached for only 4 cases out of 16. Psychiatrists did not reach a higher level of agreement on the ratings of dangerousness than non-psychiatrists. Psychiatrists had a tendency to rate individuals as more dangerous than did non-psychiatrists. The results of this exercise do not support the use of 'dangerousness' as a scientifically or operationally valid concept.

Risk assessment for people with mental health problems: a pilot study of reliability in working practice

“…For example, Montandon and Harding asked 193 raters to assess thè dangerousness' of 16 individual s on a four-point scale ranging from 0 (no dangerousne ss) to 3 (extreme dangerousne ss). 6 The case reports used in their study comprised individual s who were: 1. both violent and mentally ill (n=4); 2. violent but not mentally ill (n=4); 3. not violent but mentally ill (n=5); and 4. neither violent nor mentally ill (n=3). The level of agreement between the raters was generall y found to be quite poor (range 29 ± 86%), with a level of 60% being reached in only one quarter of cases; at worst, agreement only marginally exceeded chance levels (29% vs. 25%).…”

Structure and content of risk assessment proformas in mental healthcare

“…Irrespective of the professional background of the rater, the level of pair-wise rater agreement was only moderate and, in some cases, rather poor.Moreover, in a later study (Gale et al, 2003), we found that mental health professionals had profound difficulties with some simple arithmetic and probabilistic concepts that underlie risk measurement. These studies, amongst others (e.g., Montandon & Harding, 1984;Harriss & Hawton, 2005) raise important questions about the reliability and validity of risk assessment in current mental health practice.…”

Do mental health professionals really understand probability? Implications for risk assessment and evidence-based practice

“…Indeed, legislation and guidelines in the mid-1990s in the UK reinforced the importance of thorough risk assessment and management (DoH, 1994;1995;HMSO, 1995). Previous studies on the risk assessment of psychiatric patients has highlighted inter-professional variability in the levels of risk ascribed to specific cases (Montandon & Harding, 1984;Ryan, 1998). The gender of the assessor is also known to be a significant factor in determining whether ratings are more or less cautious (Ryan, 1998;Gale et al, 2002).…”