Low reliability when determining criterion a for posttraumatic stress disorder from self-report descriptions of traumatic events: The need for transparent methods.

Abstract

Objective: Exposure to a traumatic event is a primary criterion (Criterion A) for meeting Posttraumatic Stress Disorder (PTSD). Using self-report to establish diagnostic criteria in research has become more common, especially with internet-based research. However, some individuals may construe events as traumatic when they do not meet Criterion A. There has yet to be a test of interrater reliability (IRR) from self-report of traumatic events. Method: Three graduate students in clinical psychology and three licensed psychologists rated Criterion A using the life events checklist (LEC), as well as the three modified LEC versions (specification of up to three index traumas; extension of part 2 of the LEC) aimed to increase IRR. One hundred participants completed each of the four versions of the LEC (N = 400). Bootstrapped permutation tests were used to estimate differences in IRR and to generate 95% confidence intervals (CIs). Results: Overall, findings indicated fair–moderate IRR (Fleiss’s kappa) κ = 0.428, 95% CI [0.379, 0.477]. The other versions of the LEC (including additional clarifying questions in part 2 of the LEC and/or opportunities to describe up to three traumas) did not meaningfully increase IRR. Conclusions: Findings indicate that relying on self-report from the LEC alone and/or single-rater assessment of open-text trauma descriptions is not recommended for determining whether a traumatic event meets Criterion A. We conclude that it is critical when collecting self-reported PTSD symptoms to provide a clear description of how Criterion A was assessed, initial agreement between raters, and how disagreements were resolved.