Psychometric Properties of a Fidelity Scale for Illness Management and Recovery

Self Cite

Purpose Service providers need effective strategies to implement evidence-based practices (EBPs) with high fidelity. This study aimed to evaluate an intensive implementation support strategy to increase fidelity to EBP standards in treatment of patients with psychosis. Methods The study used a cluster randomized design with pairwise assignment of practices within each of 39 Norwegian mental health clinics. Each site chose two of four practices for implementation: physical health care, antipsychotic medication management, family psychoeducation, illness management and recovery. One practice was assigned to the experimental condition (toolkits, clinical training, implementation facilitation, data-based feedback) and the other to the control condition (manual only). The outcome measure was fidelity to the EBP, measured at baseline and after 6, 12, and 18 months, analyzed using linear mixed models and effect sizes. Results The increase in fidelity scores (within a range 1–5) from baseline to 18 months was significantly greater for experimental sites than for control sites for the combined four practices, with mean difference in change of 0.86 with 95% CI (0.21; 1.50), p = 0.009). Effect sizes for increase in group difference of mean fidelity scores were 2.24 for illness management and recovery, 0.68 for physical health care, 0.71 for antipsychotic medication management, and 0.27 for family psychoeducation. Most improvements occurred during the first 12 months. Conclusions Intensive implementation strategies (toolkits, clinical training, implementation facilitation, data-based feedback) over 12 months can facilitate the implementation of EBPs for psychosis treatment. The approach may be more effective for some practices than for others.

Section: Methodsmentioning

confidence: 98%

“…Table 1 shows a brief description or components of each of the four practices. Previous papers described the four practices in greater detail (Egeland et al, 2020 ; Joa et al, 2020 ; Ruud, 2020a , b ).…”

Section: Methodsmentioning

confidence: 99%

The Effect of Intensive Implementation Support on Fidelity for Four Evidence-Based Psychosis Treatments: A Cluster Randomized Trial

Ruud

Drake

Benth

et al. 2021

Self Cite

“…Another observation is that, on the whole, the psychometric properties of the scales examined were found to be satisfactory. However, the fidelity scales that have been around for some time—addressing Family Psychoeducation (FPE; Joa et al 2020 ), Illness Management and Recovery (IMR; Egeland et al 2019 ) and the General Organizational Index (GOI; Heiervang et al 2020 )—overall had slightly better results than those that have been developed more recently for physical healthcare (Ruud et al 2020a ) and antipsychotic medication management (Ruud et al 2020b ). This is probably due, at least in part, to the fact that the earlier scales have a longer history of testing and improvement.…”

Section: Fidelity Scales Examined In Norwaymentioning

confidence: 99%

“…However, the reverse was true with regard to the scale for antipsychotic medication management: the implementation of prescriber fidelity was slightly more successful than was the implementation of policy fidelity (Ruud et al 2020b ). Another intriguing question concerns why the implementation of IMR (Egeland et al 2019 ) proceeded so well and was so much more successful than was the case for the other interventions. This might have been due to the characteristics of the intervention, the expertise and motivation of the practitioners involved, or it might have had more to do with organizational conditions.…”

Section: Fidelity Scales Examined In Norwaymentioning

confidence: 99%

Fidelity Scale: From Black Box to Holy Grail

Weeghel

2020

Fidelity scales are indispensable in the pursuit of evidence-based mental healthcare. Without fidelity checks, treatment remains a mysterious black box. The aim of this article is to comment on the studies in this special section, and to discuss some general issues with regard to fidelity assessment. Despite all of their supposed benefits, resistance to fidelity scales persists among mental health practitioners. One way to overcome this resistance is to conduct fidelity assessments in the context of a well-guided learning community. The predictive validity of fidelity scales is considered the single most valuable attribute of these instruments. Research on predictive validity requires large sample sizes, which is difficult to achieve. It should nevertheless not prevent us from rigorously searching for this Holy Grail of fidelity assessment. In addition, fidelity assessment should be placed in a broader perspective. The quality of care for people with severe mental illness cannot be assessed conclusively according to the extent to which separate interventions have been applied with good fidelity. These individuals need access to high-quality treatment and support systems within the community, which can enable them to live their lives as valued citizens. In conclusion, fidelity assessment, both at the level of interventions and systems, contributes to a highly desirable transparency in practice variations within the field of mental healthcare.

“…All five scales followed the guidelines for fidelity scale development described by Bond and Drake (2019). Egeland et al (2019) report on the Fidelity Scale for Illness Management and Recovery (IMR), a scale developed in the National EBP Project (McHugo et al 2007) and subsequently used in several studies. The paper by Egeland et al is the first comprehensive assessment of the psychometric properties of the scale.…”

Section: Papers In the Special Sectionmentioning

confidence: 99%

Measuring Fidelity to Evidence-Based Practices: Psychometrics

Ruud

Drake

Bond

2020

This special section presents the psychometric properties of fidelity scales used in a national mental health services project in Norway to improve the quality of care of people with psychoses. Across Norway, 39 clinical units in six health trusts participated. The project provided education, implementation support and fidelity assessments. The papers in the section address the psychometrics of fidelity measurement for the specific evidence-based practices: illness management and recovery, family psychoeducation, physical healthcare and antipsychotic medication management. Another paper analyzes the psychometrics of a scale measuring individualization and quality improvement that may be used in conjunction with fidelity scales for specific evidence-based practices. The first paper in the section presents the development and field of fidelity scales, and the two final papers with comments add some additional perspectives and discuss fidelity scales in a wider context. The psychometrics of the five scales were good to excellent. Fidelity assessment is a necessary and effective strategy for quality improvement.