The use of case vignettes in studies of interrater reliability of psychiatric target syndromes and diagnoses

Abstract

This study is part of the ICD-10 field trials in which the use of case vignettes for interrater agreement has been examined. From our electronic database of 880 consecutively admitted inpatients we selected 24 cases that were transcribed to vignettes covering the first 5 ICD-10 target syndrome of dementia, substance use disorders, schizophrenia, mood and anxiety disorders. ICD-10 was compared with ICD-8 and DSM-III. The results showed that all 3 standard classification systems obtained an acceptable interrater agreement. Among the diagnoses, depressive disorders gave rise to most disagreement between the raters. Discrepancies between the methods of measuring interrater agreement were found when intraclass reliability was compared with consensus calculations for the individual patient.

Problem Solving Style and Coping Strategies: Effects of Perceived Stress

“…As mentioned previously, case vignettes have successfully been used to change mood perception in a variety of research studies (Hjortso et al, 1989). The importance of this result in the present study is that it supports the view that participants were attending to the directions given by this researcher, and that, by inference, were providing responses in a serious and truthful manner.…”

Interrater Agreement Among Psychiatrists in Psychiatric Emergency Assessments

“…Several studies have investigated the reliability of psychiatric diagnosis (8)(9)(10)(11)(12)(13)(14); many of these reported low interrater reliability. As an example, Hjortso et al (9) created 24 vignettes based on clinical records and asked seven experienced psychiatrists to make diagnostic judgments. Agreement coefficients were 0.55 for psychiatric syndromes such as schizophrenia, 0.52 for personality disorders, 0.66 for psychosocial stressors, and 0.47 for global functioning.…”

Patient impairment lexicon: A validation study

“…This was accomplished by carefully constructing 20 prototypical analogue "patients," each portrayed in a separate vignette, who collectively depicted all 63 impairments. Previous research supports vignette use for psychometric assessment of diagnostic systems (e.g., Andersen & Hawthorn, 1989;Hasin & Link, 1988;Heumann & Morey, 1990; Hjortso, S., Butler, B., Clemmensen, L., Jepsen, P. W., Kastrup, M., Vilmar, T., & Bech, P., 1989;Knesper, Pagnucco, & Kalter, 1986;Marwit, 1996;Shepherd, Brooke, Cooper, & Lin, 1968;Sorensen, Hargreaves, & Frielander, 1982). Vignettes studies increase the standardization of validity assessment, with all raters examining identical clinical data.…”

“…Given the low incidence of some patient dysfunctions, however, this would require hundreds-if not thousands-of patients to insure sufficient data to test each of the 63 impairments. Thus, whereas vignettes obviously limit generalizability of findings, their use represents a cost-effective primary evaluation of a classification system (Hjortso et al, 1989).…”