Assessing the need of young people using online counselling services: how useful are standardised measures?

Abstract

Background Clinical assessments for children and young people entering a mental health service help to identify the prevalence of need within that population, support intervention recommendations, and enable service evaluation. Evidence related to the use of standardised measures in an ever‐expanding online environment, for the purpose of identifying need, is limited. Methods This study explores the reliability of using a standardised measure to detect clinical need in an online therapeutic environment, and the measures assessed are as follows: Strengths and Difficulties Questionnaire (SDQ), Young Person’s CORE (YP‐CORE) and the Short Warwick and Edinburgh Wellbeing Scale (SWEMWBS). A deep‐dive approach is used to inform practitioner assessment of young people, followed by a Weighted Cohen’s Kappa (Κw) to measure the interrater reliability between this and the individuals’ self‐rated outcome. Composite case studies represent the complexities of presentation among the sample population. Results The interrater reliability between self‐rated and practitioner rated assessment varied between Κw = .222 and Κw = 0.446 depending on the measure. High levels of need and low levels of well‐being were found among the sample (YP‐CORE Avg. = 26.9, SDQ Avg. = 19.56, SWEMWBS Avg. = 18.1). Conclusions The findings demonstrate a fair to moderate reliability when assessing concordance between service users and practitioners, which suggests standardised measures are a reliable indicator of need. Higher levels of need were present than those seen previously in general or face‐to‐face clinical populations, which suggests using such measures in an online therapeutic environment influences the way in which assessments are responded to.

The Session Wants and Need Outcome Measure: The Development of a Brief Outcome Measure for Single-Sessions of Web-Based Support

“…Nomothetic PROMs are not always tested or designed with the digital mental health provision in mind, and they are often reported to be alienating by children and young people (Sharples et al, 2017). Furthermore, in an online therapeutic context, service users have been shown to score higher overall on nomothetic measures than the clinical population (Mindel et al, 2021).…”

Exploring the experiences of young people completing the YP‐CORE to capture clinical need on a digital counselling platform

“…While we're not able to confirm that every account of responses would be an honest one, our findings suggest that assessment data collected online is a true reflection of the individual's experiences of their need. This suggests that higher rates of need found within online populations are likely to be as a result of another online factor rather than over-catastrophe or exaggeration of response (Mindel et al, 2021). As demonstrated through the varying emotional states our participants recalled to have been in, it is unknown what emotional state an individual will present with when they come to register for, or use, an online service (Epstein & Klinkenberg, 2001).…”

“…Reliability of the data collected through online counselling services is central to supporting clinical practice and contributing to population-level data (Sharples et al, 2016). With the use of standardised mental health assessment measures indicating a greater level of need when collected online (Mindel et al, 2021), the question of reliability and validity of results is ever more crucial when considering the transference of measures validated in face-to-face (f2f) settings…”

“…The concerns and benefits related to self-report assessments mentioned above have been identified within either f2f settings, or using computers on-site with the intervention. There is little research exploring both remote usage of standardised mental health assessment measures, and the reasons the use of such a measure elicits a higher level of need than found in f2f settings (Mindel et al, 2021). One theory underpinning the evidence for greater need online is the disinhibition effect, whereby people appear to feel freer in their disclosure as a result of the online environment (Suler, 2005).…”

Digital mental health outcome monitoring for a structured text‐based youth counselling intervention: Demographic profile and outcome change

“…Furthermore, for text‐based mental health interventions, regular human‐guided support has been shown to be more effective than machine or automatically guided content (Koelen et al, 2022) and their qualifications do not make them more efficacious as practitioners. Far less research was identified about the value of online interventions for young people in real‐world implementation or person‐centred approaches more broadly, perhaps owing to a lack of consistent measures or the delivery of this kind of support in online therapeutic environments (Mindel et al, 2021). For online and Internet‐delivered interventions the methodological tradition and preference for building an evidence base has been through experimental designs and Randomised Control Trials (RCT; Mohr et al, 2017).…”