Assessing organizational implementation context in the education sector: confirmatory factor analysis of measures of implementation leadership, climate, and citizenship

Abstract

BackgroundA substantial literature has established the role of the inner organizational setting on the implementation of evidence-based practices in community contexts, but very little of this research has been extended to the education sector, one of the most common settings for the delivery of mental and behavioral health services to children and adolescents. The current study examined the factor structure, psychometric properties, and interrelations of an adapted set of pragmatic organizational instruments measuring key aspects of the organizational implementation context in schools: (1) strategic implementation leadership, (2) strategic implementation climate, and (3) implementation citizenship behavior.MethodThe Implementation Leadership Scale (ILS), Implementation Climate Scale (ICS), and Implementation Citizenship Behavior Scale (ICBS) were adapted by a research team that included the original scale authors and experts in the implementation of evidence-based practices in schools. These instruments were then administered to a geographically representative sample (n = 196) of school-based mental/behavioral health consultants to assess the reliability and structural validity via a series of confirmatory factor analyses.ResultsOverall, the original factor structures for the ILS, ICS, and ICBS were confirmed in the current sample. The one exception was poor functioning of the Rewards subscale of the ICS, which was removed in the final ICS model. Correlations among the revised measures, evaluated as part of an overarching model of the organizational implementation context, indicated both unique and shared variance.ConclusionsThe current analyses suggest strong applicability of the revised instruments to implementation of evidence-based mental and behavioral practices in the education sector. The one poorly functioning subscale (Rewards on the ICS) was attributed to typical educational policies that do not allow for individual financial incentives to personnel. Potential directions for future expansion, revision, and application of the instruments in schools are discussed.

EBP Implementation Leadership of Frontline Nurse Managers: Validation of the Implementation Leadership Scale in Acute Care

“…Overall, the strength of these findings is noteworthy given that data were collected independently, by two different investigative teams, in very different geographic areas of the U.S., and with broadly different nurse demographics. Our results contribute to the strong record of the scale’s validity and reliability in other contexts (Aarons et al, ; Aarons et al, ; Finn et al, ; Lyon et al, ; Torres et al, ).…”

“…The scale focuses on specific leadership behaviors enacted by frontline managers to influence implementation, including: (a) proactive leadership; (b) knowledgeable leadership; (c) supportive leadership; and (d) perseverant leadership. The ILS, originally developed and tested in mental health settings, has been cross‐validated in substance use facilities (Aarons, Ehrhart, Torres, Finn, & Roesch, ), child welfare service organizations (Finn, Torres, Ehrhart, Roesch, & Aarons, ), and education sectors (Lyon et al, ). However, it has yet to be validated for use in nursing contexts.…”

Implementing a State‐Adopted High School Health Curriculum: A Case Study

“…These results suggest that this type of institutional support for evidence‐based curricula may be especially important for delivering EBPs with high fidelity. Furthermore, our results are consistent with a recent study by Lyon and colleagues, who found that a general focus on EBPs may be an important support for effective delivery . Although general support of EBPs seems to be associated with fidelity, our interview results point to less specific support for health education.…”

“…We found an association between focus on EBP and fidelity. A growing body of research recognizes the critical role implementation climate plays in the successful implementation of programs, including programs in schools . These results suggest that this type of institutional support for evidence‐based curricula may be especially important for delivering EBPs with high fidelity.…”

Implementation leadership: A concept analysis

“…It was not clear whether this was in reference to general attitudes towards EBPs or for an identified practice change. Additionally, non‐nursing studies made mention of the impact of IL on implementation citizenship (Aarons, Ehrhart, Torres, et al, ; Finn, Torres, Ehrhart, Roesch, & Aarons, ; Lyon et al, ; Powell et al, ), which are the behaviours by staff that support EBP. These studies proposed implementation citizenship as a mediating variable between IL and implementation climate.…”