Tools for assessing the scalability of innovations in health: a systematic review

Abstract

Background The last decade has seen growing interest in scaling up of innovations to strengthen healthcare systems. However, the lack of appropriate methods for determining their potential for scale-up is an unfortunate global handicap. Thus, we aimed to review tools proposed for assessing the scalability of innovations in health. Methods We conducted a systematic review following the COSMIN methodology. We included any empirical research which aimed to investigate the creation, validation or interpretability of a scalability assessment tool in health. We searched Embase, MEDLINE, CINAHL, Web of Science, PsycINFO, Cochrane Library and ERIC from their inception to 20 March 2019. We also searched relevant websites, screened the reference lists of relevant reports and consulted experts in the field. Two reviewers independently selected and extracted eligible reports and assessed the methodological quality of tools. We summarized data using a narrative approach involving thematic syntheses and descriptive statistics. Results We identified 31 reports describing 21 tools. Types of tools included criteria (47.6%), scales (33.3%) and checklists (19.0%). Most tools were published from 2010 onwards (90.5%), in open-access sources (85.7%) and funded by governmental or nongovernmental organizations (76.2%). All tools were in English; four were translated into French or Spanish (19.0%). Tool creation involved single (23.8%) or multiple (19.0%) types of stakeholders, or stakeholder involvement was not reported (57.1%). No studies reported involving patients or the public, or reported the sex of tool creators. Tools were created for use in high-income countries (28.6%), low- or middle-income countries (19.0%), or both (9.5%), or for transferring innovations from low- or middle-income countries to high-income countries (4.8%). Healthcare levels included public or population health (47.6%), primary healthcare (33.3%) and home care (4.8%). Most tools provided limited information on content validity (85.7%), and none reported on other measurement properties. The methodological quality of tools was deemed inadequate (61.9%) or doubtful (38.1%). Conclusions We inventoried tools for assessing the scalability of innovations in health. Existing tools are as yet of limited utility for assessing scalability in health. More work needs to be done to establish key psychometric properties of these tools. Trial registration We registered this review with PROSPERO (identifier: CRD42019107095)

Feasibility and scalability of a fitness tracker study: Results from a longitudinal analysis of persons with multiple sclerosis

“…Our findings should be considered formative and interpreted with appropriate caution. Furthermore, although the ISAT tool provided a helpful framework for our scalability assessments, it was not specifically designed for mHealth studies and recently has been qualified to require further validation by a systematic review (25). Therefore, relevant scalability elements or axes could have been missed by our analysis.…”

“…These and other factors also form the foundation for the Intervention Scalability Assessment Tool (ISAT) tool that notably examines implementation and scale-up potential on five axes: (1) "fidelity and adaptation", (2) "reach and acceptability", (3) "delivery setting and workforce", (4) "implementation infrastructure", and ( 5) "sustainability" (24). The ISAT tool, along with similar other checklists (25), helps to assess the readiness interventions for a later scale-up.…”

Investments for effective functionality of health systems towards Universal Health Coverage in Africa: A scoping review

“…ABC drafted the preliminary version of the search strategy for Ovid MEDLINE. The search terms were based on previous works to reflect three concepts: 1) intervention [27,28], 2) health system functionality [11,29], and 3) low-and middle-income regions [30,31]. For the latter, we used a search filter developed by the Cochrane Effective Practice and Organisation of Care (EPOC) in collaboration with the WHO and Campbell Collaboration [30,31].…”

“…Third, there is widespread enthusiasm for scaling effective interventions to improve functionality of African health systems [11,27], but we know little about the scalability of identified interventions. By scalability, we mean the ability of an effective intervention to change in size, while retaining effectiveness [94,95].…”

“…Thus, there is a need for a tool to help rate and rank our included interventions for their scalability across African countries. A systematic review published a useful inventory of tools, components, and items for assessing the scalability of interventions in health as well as interpretability criteria that could be used to weed out poor items [27]. This will help to propose a scalability assessment tool aligned with the Framework of Actions and to select effective interventions that could be successfully utilized to improve the functionality of health systems in African countries.…”