Provider REport of Sustainment Scale (PRESS): development and validation of a brief measure of inner context sustainment

Abstract

Background Implementation scientists and practitioners often rely on frontline providers for reporting on implementation outcomes. Furthermore, measures of sustainment are few, and available sustainment measures are mainly setting or evidenced-based practice (EBP) specific, require organizational and system-level knowledge to complete, and often lack psychometric rigor. The aim of this study was to develop a brief, pragmatic, and generalizable measure for completion by frontline service providers of the implementation outcome, sustainment. Methods We utilized a Rasch measurement theory approach to scale the development and testing of psychometric parameters. Sustainment items were developed to be relevant for direct service providers to complete. In order to promote generalizability, data were collected and items were tested across four diverse psychosocial evidence-based practices (motivational interviewing [MI], SafeCare®, classroom pivotal response training [CPRT], and an individualized mental health intervention for children with autism spectrum disorder [AIM-HI]) and in four service settings (substance use disorder treatment, child welfare, education, and specialty mental health). Associations between the sustainment measure and sustainment leadership, sustainment climate, and attitudes towards the adoption and use of each of the EBPs were assessed to confirm construct validity. Results Three items for the Provider REport of Sustainment Scale (PRESS) were assessed for measuring the core component of sustainment: continued use of the EBP. Internal consistency reliability was high. The scale indicated fit to the Rasch measurement model with no response dependency, ordered thresholds, no differential item functioning, and supported unidimensionality. Additionally, construct validity evidence was provided based on the correlations with related variables. Conclusion The PRESS measure is a brief, three-item measure of sustainment that is both pragmatic and useable across different EBPs, provider types, and settings. The PRESS captures frontline staffs’ report of their clinic, team, or agency’s continued use of an EBP. Future testing of the PRESS for concurrent and predictive validity is recommended.

Evaluation of measures of sustainability and sustainability determinants for use in community, public health, and clinical settings: a systematic review

“…This is likely due to most of the measure assessing more specific constructs or aspects of sustainability, rather than the broader definition of sustainability used by Moore. For instance, sustainment has been recognised as a distinct concept, defined as the ongoing delivery of an evidence-based intervention [ 2 , 8 , 11 , 32 ] and which was the focus of some of the measures included in this review, including PRESS [ 32 ]. As we were attempting to provide a comprehensive review of all quantitative measures related to sustainability we took a broad definition and included any related measures to sustainability.…”

Sustainment of diverse evidence-informed practices disseminated in the Veterans Health Administration (VHA): initial development and piloting of a pragmatic survey tool

“…Staff continue to use [EIP] throughout changing circumstances; 3. [EIP] is a routine part of our practice [ 39 ]. To our knowledge, the PRESS is the first validated pragmatic sustainment instrument and addresses key issues highlighted by Moullin et al in their previous narrative review [ 21 ] (see Background: Limitations to current instruments).…”

“…Although we tried to limit bias, this is a real-world quality improvement project, and there are several limitations that may have skewed our results: (1) the use of self-report, potential social desirability, and lack of fidelity assessment; 2) the size of our sample; and (3) the rate of missing data. Regarding the first limitation: Although self-report may be less objective than in-person observation, it is commonly used to assess sustainment (e.g., both the SMSS [ 23 ] and PRESS [ 39 ] rely on self-report), because it is more pragmatic and feasible [ 39 ]. Given travel and funding limitations, self-report was the only option in our project.…”

“…Staff continue to use [EIP] throughout changing circumstances; 3. [EIP] is a routine part of our practice[39]. To our knowledge, the PRESS is the first validated pragmatic sustainment instrument and addresses key…”

Iterative delivery of an implementation support package to increase and sustain the routine provision of antenatal care addressing alcohol consumption during pregnancy: study protocol for a stepped-wedge cluster trial

“…Acceptability of the strategies will be measured from the perspective of maternity staff. Sustainment at the provider and inner-context levels will be measured from the perspective of maternity staff using the three-item Provider REport of Sustainment Scale 57. Changes occurring at the outer contextual level (eg, social, political and economic factors) that may influence practices will be monitored and reported.…”