Assessing the maturity of the healthcare system for integrated care: testing measurement properties of the SCIROCCO tool

Abstract

BackgroundThe Scaling Integrated Care in Context (SCIROCCO) tool has been developed to facilitate knowledge transfer and learning about the implementation and scaling-up of integrated care in European regions. To adequately test the functionality of the tool in assessing the maturity for integrated care within regions, this study evaluated its structural validity, internal consistency and convergent validity.MethodsExploratory factor analysis was used to investigate the structural validity of the 12-items of the SCIROCCO tool. Hereafter, the internal consistency was assessed by calculating Cronbach’s and ordinal alpha. The convergent validity was explored by testing 23 pre-hypothesized relationships between items of the SCIROCCO tool and items of an instrument measuring a similar construct.ResultsFactor analysis revealed a one-factor structure. Cronbach’s alpha of the overall instrument was 0.92, ordinal alpha was 0.94. Only 30.34% of the hypotheses for testing the convergent validity were met.ConclusionThe one-factor structure is considered relevant in representing the structural validity of the SCIROCCO tool. The scale of the SCIROCCO tool shows good internal consistency. The tool (DMIC Quickscan) used to assess the convergent validity might measure a different aspect of integrated care than the SCIROCCO tool. Further research is needed to continue investigating the validity and reliability of the tool.Electronic supplementary materialThe online version of this article (10.1186/s12874-019-0704-1) contains supplementary material, which is available to authorized users.

Implementation outcome instruments for use in physical healthcare settings: a systematic review

“…The majority of included studies reported measurement properties of instruments that assessed acceptability ( n = 33) [ 23 – 59 ], followed by appropriateness ( n = 7) [ 46 , 60 – 65 ], adoption ( n = 4) [ 66 – 69 ], feasibility ( n = 4) [ 46 , 70 , 71 , 80 ], penetration ( n = 4) [ 72 – 75 ] and sustainability of evidence-based practice (intervention or service) ( n = 3) [ 76 – 79 ]. No studies of intruments measuring implementation cost were identified (see Fig.…”

“…Organisation-level analysis was observed for scales assessing penetration and sustainability, though the overall number of these scales was small. Summary of level of analysis across outcomes as follows: Acceptability: consumer = 10 [ 23 , 37 , 39 , 41 , 42 , 44 , 51 , 52 , 57 , 59 ], provider = 27 [ 24 – 36 , 38 , 40 , 43 , 45 – 50 , 53 – 56 , 58 ] Appropriateness: consumer = 3 [ 61 – 63 ], provider = 3 [ 46 , 60 , 64 ], both consumer and provider = 1 [ 65 ] Adoption: consumer = 1 [ 66 ], provider = 3 [ 67 – 69 ] Feasibility: provider = 4 [ 46 , 70 , 71 , 80 ] Penetration: organisation = 4 [ 72 – 75 ] Sustainability: provider = 2 [ 76 , 77 ], organisation = 2 [ 78 , 79 ] …”

“…Commonly used terms: actual fit or utility, suitability for everyday use and practicability Garcia-Smith et al [ 70 ] Instrument to test the Clinical Information Systems Success Model (CISSM) Excellent Poor Not assessed Good (26) 2 Schnall et al [ 71 ] Technology Acceptance Survey Fair Poor Not assessed Excellent (8) 3 Windsor et al (2013) The Smoking Cessation and Reduction in Pregnancy Treatment (SCRIPT) Adoption Scale Poor Poor Not assessed Good (28) 4 Weiner et al [ 46 ]** Feasibility of Intervention Measure (FIM) Poor Poor Not assessed Excellent (4) 8 Penetration (number of instruments = 4): the integration of a practice within a service setting and its subsystems. Commonly used terms: level of institutionalisation, spread and service access Grooten et al [ 72 ] The Scaling Integrated Care in Context (SCIROCCO) tool Good Good Not assessed Good (12) 5 Slaghuis et al [ 73 ] A measurement instrument for spread of quality improvement in healthcare Fair Poor Not assessed Good (18) 4 Flanagan et al [ 74 ] The Prevention and Control of Antimicrobial resistance (PACAR) scale Poor Fair Not assessed Good (16) 6 Jaana et al [ 75 ] A measure of clinical information technology sophistication in hospitals Poor Not assessed Not assessed Adequate (68) 3 Sustainability (number of instruments = 3): the extent to which a newly implemented treatment is maintained or institutionalised within a service setting’s on...…”

“…Acceptability: consumer = 10 [23,37,39,41,42,44,51,52,57,59], provider = 27 [24-36, 38, 40, 43, 45-50, 53-56, 58] Appropriateness: consumer = 3 [61][62][63], provider = 3 [46,60,64], both consumer and provider = 1 [65] Adoption: consumer = 1 [66], provider = 3 [67][68][69] Feasibility: provider = 4 [46,70,71,80] Penetration: organisation = 4 [72][73][74][75] Sustainability: provider = 2 [76,77], organisation = 2 [78,79] Methodological quality Table 1 presents the global COSMIN scores for reliability, validity and responsiveness, and rank orders the instruments by descending reliability scores. Detailed COSMIN scores for individual measurement properties are presented in Additional file 3.…”

Mixed-method evaluation of CARITAS: a hospital-to-community model of integrated care for dementia

“…Integrated care programmes evolve with time and some dimensions mature more quickly than others. 36 Integration often begins at micro-levels (eg, clinical integration) and meso-levels (eg, professional and organisational integration) before progressing to a macro-level (eg, system integration). 37 Dimensions such as functional and normative integration which establish connectivity across the micro, meso and macro require significant time to stabilise.…”