Authorship Bias in Violence Risk Assessment? A Systematic Review and Meta-Analysis

Abstract

Various financial and non-financial conflicts of interests have been shown to influence the reporting of research findings, particularly in clinical medicine. In this study, we examine whether this extends to prognostic instruments designed to assess violence risk. Such instruments have increasingly become a routine part of clinical practice in mental health and criminal justice settings. The present meta-analysis investigated whether an authorship effect exists in the violence risk assessment literature by comparing predictive accuracy outcomes in studies where the individuals who designed these instruments were study authors with independent investigations. A systematic search from 1966 to 2011 was conducted using PsycINFO, EMBASE, MEDLINE, and US National Criminal Justice Reference Service Abstracts to identify predictive validity studies for the nine most commonly used risk assessment tools. Tabular data from 83 studies comprising 104 samples was collected, information on two-thirds of which was received directly from study authors for the review. Random effects subgroup analysis and metaregression were used to explore evidence of an authorship effect. We found a substantial and statistically significant authorship effect. Overall, studies authored by tool designers reported predictive validity findings around two times higher those of investigations reported by independent authors (DOR = 6.22 [95% CI = 4.68–8.26] in designers' studies vs. DOR = 3.08 [95% CI = 2.45–3.88] in independent studies). As there was evidence of an authorship effect, we also examined disclosure rates. None of the 25 studies where tool designers or translators were also study authors published a conflict of interest statement to that effect, despite a number of journals requiring that potential conflicts be disclosed. The field of risk assessment would benefit from routine disclosure and registration of research studies. The extent to which similar conflict of interests exists in those developing risk assessment guidelines and providing expert testimony needs clarification.

Memory detection with the Concealed Information Test: A meta analysis of skin conductance, respiration, heart rate, and P300 data

“…When these studies were eliminated, publication year was no longer correlated with the effect size and had no effect in the multiple regression analysis. Interestingly, the fact that authors who design a given test find higher validity estimates than those reported in subsequent studies has been shown before, not only for psychophysiological deception detection tests (Krapohl, ), but also for prognostic instruments designed to assess violence risk (Singh, Grann, & Fazel, ).…”

Risk of Recidivism Among Justice-Involved Veterans

“…From a purely empirical standpoint, it is widely accepted that (a) a risk assessment should be structured and informed by empirically supported risk cues (Dawson, 2000;Elbogen et al, 2010), and (b) actuarial approaches are superior to unstructured clinical judgment in terms of predicting recidivism (Grove et al, 2000;Hanson & Morton-Bourgon, 2009). Nonetheless, recent work has called into question whether the evidence supports the sole use of actuarial tools in the prediction of recidivism risk (Fazel et al, 2012;Singh et al, 2013). Furthermore, team-based models that encompass shared assessments of patient risk and strive to establish agreement across multiple providers have been shown to reduce the unreliability of clinical judgment and improve predictive accuracy of recidivism outcomes (Campbell, 1995;Elbogen, 2002).…”

Treatment of Psychosis and Risk Assessment for Violence

“…In replication studies, existing risk instruments can identify groups of high-risk patients who are approximately three times more likely to be violent than low-risk patients (15). In replication studies, existing risk instruments can identify groups of high-risk patients who are approximately three times more likely to be violent than low-risk patients (15).…”