Allegiance effects in assessment: Unresolved questions, potential explanations, and constructive remedies.

Lilienfeld, Scott O.; Jones, Meredith

doi:10.1111/j.1468-2850.2008.00148.x

Cited by 8 publications

(17 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous work has proposed several possible explanations of such bias [7], [9], [45]. First, tool designers may conduct studies to maximize the predictive validity of their instruments.…”

Section: Discussionmentioning

confidence: 99%

Authorship Bias in Violence Risk Assessment? A Systematic Review and Meta-Analysis

2013

View full text Add to dashboard Cite

Various financial and non-financial conflicts of interests have been shown to influence the reporting of research findings, particularly in clinical medicine. In this study, we examine whether this extends to prognostic instruments designed to assess violence risk. Such instruments have increasingly become a routine part of clinical practice in mental health and criminal justice settings. The present meta-analysis investigated whether an authorship effect exists in the violence risk assessment literature by comparing predictive accuracy outcomes in studies where the individuals who designed these instruments were study authors with independent investigations. A systematic search from 1966 to 2011 was conducted using PsycINFO, EMBASE, MEDLINE, and US National Criminal Justice Reference Service Abstracts to identify predictive validity studies for the nine most commonly used risk assessment tools. Tabular data from 83 studies comprising 104 samples was collected, information on two-thirds of which was received directly from study authors for the review. Random effects subgroup analysis and metaregression were used to explore evidence of an authorship effect. We found a substantial and statistically significant authorship effect. Overall, studies authored by tool designers reported predictive validity findings around two times higher those of investigations reported by independent authors (DOR = 6.22 [95% CI = 4.68–8.26] in designers' studies vs. DOR = 3.08 [95% CI = 2.45–3.88] in independent studies). As there was evidence of an authorship effect, we also examined disclosure rates. None of the 25 studies where tool designers or translators were also study authors published a conflict of interest statement to that effect, despite a number of journals requiring that potential conflicts be disclosed. The field of risk assessment would benefit from routine disclosure and registration of research studies. The extent to which similar conflict of interests exists in those developing risk assessment guidelines and providing expert testimony needs clarification.

show abstract

“…Previous work has proposed several possible explanations of such bias [7], [9], [45]. First, tool designers may conduct studies to maximize the predictive validity of their instruments.…”

Section: Discussionmentioning

confidence: 99%

Authorship Bias in Violence Risk Assessment? A Systematic Review and Meta-Analysis

2013

View full text Add to dashboard Cite

show abstract

“…The accompanying commentary (Lilienfeld & Jones, 2008), while reserving judgment as to whether this allegiance would be observed with other assessments, suggested but did not test other explanations for the findings of Blair et al. (2008) and for allegiance effects in general.…”

mentioning

confidence: 93%

“… [Clin Psychol Sci Prac 17: 82–89, 2010] Recently, in the journal Clinical Psychology: Science and Practice , there appeared a systematic review (Blair, Marcus, & Boccaccini, 2008) accompanied by a commentary (Lilienfeld & Jones, 2008) suggesting an “allegiance effect” in the reporting of the predictive accuracy of actuarial risk assessment systems. The authors of these two articles suggested some possible errors or misrepresentation on the part of original developers or other researchers and proposed some remedies.…”

mentioning

confidence: 99%

“…Recently, in the pages of Clinical Psychology: Science and Practice , there appeared a meta‐analysis accompanied by a commentary (Blair, Marcus, & Boccaccini, 2008; Lilienfeld & Jones, 2008) suggesting an “allegiance effect” in the reporting of empirical studies of actuarial risk assessments, such as those we developed (Quinsey, Harris, Rice, & Cormier, 2006). These instruments of ours, the Violence Risk Appraisal Guide (VRAG) and Sex Offender Risk Appraisal Guide (SORAG), clearly predicted relevant outcomes (Blair et al., 2008), and no superior method for assessing the risk of violence among forensic cases was proposed.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Allegiance or fidelity? A clarifying reply.

Harris¹,

Rice²,

Quinsey³

2010

Clinical Psychology: Science and Practice

View full text Add to dashboard Cite

[Clin Psychol Sci Prac 17: 82–89, 2010] Recently, in the journal Clinical Psychology: Science and Practice, there appeared a systematic review (Blair, Marcus, & Boccaccini, 2008) accompanied by a commentary (Lilienfeld & Jones, 2008) suggesting an “allegiance effect” in the reporting of the predictive accuracy of actuarial risk assessment systems. The authors of these two articles suggested some possible errors or misrepresentation on the part of original developers or other researchers and proposed some remedies. We examined these two articles in conjunction with all the available evidence for the Violence Risk Appraisal Guide and Sex Offender Risk Appraisal Guide and concluded there is no evidence of an allegiance effect.

show abstract

“…The limited group of investigators raises issues about ''allegiance effects,'' known for their potential to bias outcome studies (Lilienfeld & Jones, 2008). In addition, obstacles placed in the way of independent researchers make it impossible for the highest levels of replicated research evidence to be produced in support of COS, and these obstacles are presented both by commercialization and by failure to publish several documents basic to COS.…”

Section: Discussion and Applications To Social Workmentioning

confidence: 99%

Examining Circle of Security™

Mercer

2014

Research on Social Work Practice

View full text Add to dashboard Cite

The Circle of Security™ interventions are psychosocial treatments intended to increase maternal sensitivity and thus child attachment security in infants and young children. A small number of publications have reported empirical research on outcomes of these treatments. This article reviews the research evidence, plausibility, theoretical background, and generalizability of Circle of Security, and concludes that these interventions are plausible emerging treatments, presently weakly supported by research evidence, but potentially able to receive support from further research. The commercialization of Circle of Security is discussed as a possible obstacle to independent research. The current level of research support is described as similar to support for other interventions for young children.

show abstract

Allegiance effects in assessment: Unresolved questions, potential explanations, and constructive remedies.

Cited by 8 publications

References 22 publications

Authorship Bias in Violence Risk Assessment? A Systematic Review and Meta-Analysis

Authorship Bias in Violence Risk Assessment? A Systematic Review and Meta-Analysis

Allegiance or fidelity? A clarifying reply.

Examining Circle of Security™

Contact Info

Product

Resources

About