Predictive validity and interrater reliability of the FACE‐CARAS toolkit in a CAMHS setting

Abstract

Background The FACE‐CARAS (Functional Analysis in Care Environments‐Child and Adolescent Risk‐Assessment Suite) toolkit has been developed to support practitioners in Child and Adolescent Mental Health Services (CAMHS) in performing a structured risk assessment. It covers a number of risk domains including violence, suicide, self‐harm, experienced abuse, and exploitation. Interrater and internal reliability has already been established but not predictive validity. Aims/Hypothesis Our aim was to establish the predictive validity of the FACE‐CARAS in a CAMHS population. Methods Records from 123 young people with FACE‐CARAS ratings completed by clinicians were examined in a retrospective file review to extract data on a relevant list of adverse outcomes at three and at 6 months following the assessment. Although this was not a prospective longitudinal study, researchers were blind to the clinicians' ratings, allowing valid testing of predictive power. Cases were drawn from across generic and specialist CAMHS teams in approximately equal proportions. Data were analysed using receiver operator characteristic statistics. Results Areas under the curve values in five of the seven risk areas approached or were greater than 0.8 indicated that the FACE‐CARAS profile score was a good potential predictor of risks of self‐harm, suicidal behaviours, serious self‐neglect, abuse or exploitation by others, and violence to others at both 3 and 6 months. It was weakly “predictive” of accidental self‐harm and no better than chance at signalling physical ill health. Conclusions Findings support the use of the “profile summary” section of the tool as likely to generate clinically useful risk predictions. We were concerned that clinical use of the scale did not conform to research standards and often left subscales incompletely rated; however, the fact that the tool nonetheless proved a good predictor of most key adversities under scrutiny may add weight to its value in clinical practice. Further work with the FACE‐CARAS subscales is recommended.

Factors within the clinical encounter that impact upon risk assessment within child and adolescent mental health services: a rapid realist synthesis

“…5,8 The Functional Analysis of Care Environments-Child and Adolescent Risk-Assessment Suite (FACE-CARAS) suite of tools promotes use of schedules that enquire about both historical (static) and current (dynamic) risk factors. 65 Key to risk assessment is a collaborative dialogue, which encompasses the provision of adequate, detailed information across all aspects of a young person's care, including treatment options and confidentiality. 59 Assessment tends to focus on risks people with mental health diagnoses pose, which marginalises consideration of other risks like living in inadequate accommodation.…”

“…70 The FACE-CARAS tools are predicated on a stepped approach to completion -such that subsequent tools are only completed when indicated by the overall risk profile -but even within the context of research and evaluation completion of subscales was found to be unacceptably incomplete. 65 While advances in computerisation and clinical records have shifted the exact nature of this challenge the need for multiple and complementary approaches remains as pressing as ever. Specific challenges relate to conducting suicide risk assessment.…”

“…These were examined in a retrospective file review to extract data on a relevant list of adverse outcomes at 3 and at 6 months following the assessment. 65 Although this was not a prospective longitudinal study, researchers were blind to the clinicians' ratings, allowing valid testing of predictive power. The FACE-CARAS profile score was considered a good potential predictor of risks of self-harm, suicidal behaviours, serious self-neglect, abuse or exploitation by others, and violence to others at both 3 and 6 months.…”

“…The FACE-CARAS profile score was a good potential predictor of risks of self-harm, suicidal behaviours, serious self-neglect, abuse or exploitation by others, and violence to others at both 3 and 6 months. 65 It was weakly 'predictive' of accidental self-harm and no better than chance at signalling physical ill health. Clinical use of the scale did not conform to research standards and often left subscales incompletely rated.…”