Measuring implementation climate: psychometric properties of the Implementation Climate Scale (ICS) in Norwegian mental health care services

Abstract

BackgroundEmployees’ perceptions of organizational climate for implementation of new methods are important in assessing and planning for implementation efforts. More specifically, feedback from employees’ points to which implementation strategies to select, adopt, and tailor in building positive climate for implementation of new evidence-based practices within the organization. Implementation climate can be measured with the Implementation Climate Scale (ICS). The purpose of this study was to investigate the psychometric properties of the Norwegian version of the ICS in outpatient mental health clinics.MethodsThe ICS was administered to 383 clinicians within 47 different child and adult mental health clinics across the country. We conducted confirmatory factor analysis to assess the psychometric functioning of the ICS. Cronbach’s alpha was examined to assess internal consistency. We also examined criterion related validity of the scale by comparing it with an alternative measure of implementation climate (concurrent validity) and by examining correlations with clinicians’ intentions to use evidence-based practices.ResultsResults supported the 6-factor structure and the internal consistency reliability of the ICS. One exception was poor functioning of the Reward scale. Concurrent validity was stronger at the group than at the individual level, and assessment of associations with clinicians’ intentions to use evidence- based practices showed positive correlations.ConclusionsThe Norwegian version of the ICS is a promising tool for assessing implementation climate which can provide organizations with specific feedback concerning which aspects of the implementation climate to attend to. Due to poor functioning of the Reward scale, adaptations and further testing of this is recommended.

Individual-level associations between implementation leadership, climate, and anticipated outcomes: a time-lagged mediation analysis

“…A higher score indicates more favorable implementation climate. The scale showed excellent internal consistency reliability in the current study [ 41 ]. However, due to psychometric challenges as a result of financial rewards/incentives and promotions being rare in Norwegian working conditions [ 41 ], the Rewards subscale and item 9 from the Recognition subscale (more likely to be promoted) were excluded from the ICS global scale (hereby called ICS-r), resulting in 14 items.…”

“…We also acknowledge that the Rewards subscale and item 9 from the Recognition subscale were excluded from the ICS global scale due to lack of cultural relevance in the Norwegian health system context [ 41 ]. Comparisons with other studies should therefore be made cautiously.…”

Climate, Health, and Nursing Tool (CHANT): A confirmatory factor analysis

“…While CHANT began as a tool focused on nurses, it has since been expanded to include all health professionals such as physicians, dentists, pharmacists, social workers, physical therapists, psychologists, and counselors. Some efforts are being made to understand health professionals’ (Kotcher et al., 2021; Lee et al., 2021; Maibach et al., 2021) and health organizations’ (Ehrhart et al., 2016; Jagals & Ebi, 2021; Peters et al., 2022) response to climate change. Targeted action from a wide range of health professionals can help drive advocacy and change for public health prevention efforts, organizational policy and protocols, environmental justice, and environmentally safe clinical practice.…”

“…While CHANT began as a tool focused on nurses, it has since been expanded to include all health professionals such as physicians, dentists, pharmacists, social workers, physical therapists, psychologists, and counselors. Some efforts are being made to understand health professionals' Lee et al, 2021;Maibach et al, 2021) and health organizations' (Ehrhart et al, 2016;Jagals & Ebi, 2021;Peters et al, 2022) Since the data was collected for this study (Schenk et al, 2020(Schenk et al, , 2021, this tool is being used nationally and globally. We receive regular requests to translate and utilize CHANT in numerous countries and languages.…”

Dynamic interplay between available resources and implementation climate across phases of implementation: a qualitative study of a VA national population health tool

“…Understanding the users’ perspectives is essential to the implementation success as the users hold first-hand knowledge of which available resources or facilitating factors they need to better integrate the intervention into their daily work environments [ 15 , 16 ]. To assess the effectiveness of an intervention from the users’ perspectives, researchers in implementation science have turned their attention to defining and measuring implementation climate [ 17 – 20 ]. Implementation climate refers to the extent to which intended users perceive the use of the intervention as expected, supported, and rewarded within their organizations [ 21 ].…”