Multilevel investigation of variation in HoNOS ratings by mental health professionals: a naturalistic study of consecutive referrals

Abstract

Episodes of mental healthcare in specialist psychiatric services often begin with the assessment of clinical and psychosocial needs of patients by healthcare professionals. Particularly for patients with complex needs or severe problems, ratings of clinical and social functioning at the start of each episode of care may serve as a baseline against which subsequent measures can be compared. Currently, little is known about service variations in such assessments on referrals from primary care. We set out to quantify variability in initial assessments performed by healthcare professionals in three CMHTs in Bristol (UK) using the Health of the Nation Outcome Scales (HoNOS). We tested the hypothesis that variations in HoNOS total and sub-scale scores are related to referral source (general practices), healthcare assessor (in CMHTs) and the assessor's professional group. Statistical analysis was performed using multilevel variance components models with cross-classified random effects. We found that variation due to assessor substantially exceeded that due to referral source (general practices). Furthermore, patient variance differed by assessor profession for the HoNOS--Impairment scores. Assessor variance differed by assessor profession for the HoNOS--Social scores. As HoNOS total and subscale scores show much larger variation by assessor than by referral source, investigations of HoNOS scores must take assessors into account. Services should implement and evaluate interdisciplinary training to improve consistency in use of rating thresholds; such initiatives could be evaluated using these extensions of multilevel models. Future research should aim to integrate routine diagnostic data with continuous outcomes to address selection effects (of patients to assessors) better.

The reliability and validity of the forensic Camberwell Assessment of Need (CANFOR): a needs assessment for forensic mental health service users

“…In terms of making ratings as accurate and consistent as possible for both clinical and research use; some recent research has suggested that there may be differences in the perceptions of, and ratings made by, people from different professional backgrounds (Davies et al, 2006), due to differences in their clinical frames of reference, any specialist training received and levels of experience. Ecob et al (2004) suggest the need for specialized training to address any adjustments required to counter such effects. Due to the additional complexities of FMHSUs, and addressing identifi ed needs within the wider framework of risk and accountability (Andrews and Bonta, 2003;Cohen and Eastman, 2000), some formalized training about how to score CANFOR may address some of the differences between raters, and lead to greater consistency and therefore practical utility.…”

The validity of the German version of the Health of the Nation Outcome Scales (HoNOS‐D): a clinician‐rating for the differential assessment of the severity of mental disorders

“…HoNOS-D) were administered by clinicians following their initial contact with the patients during the fi rst week and the last week of inpatient treatment, just before the patients' discharge. Ecob et al (2004) pointed out that investigations of HoNOS scores must take possible interrater variation into account. In order to control for this aspect, the same clinician completed all of the clinicianrated instruments for one specifi c patient both at admission and discharge.…”

Multilevel ordinal factor analysis of the Positive and Negative Syndrome Scale (PANSS)

“…Nevertheless, many measurement tools have been developed. These measurement tools are often administered by interviewers or raters rather than being completed by the patient (Ecob et al ., ). The Positive and Negative Symptom Scale (PANSS) is an example of a scale in which interviewers rate the severity of patient symptoms.…”