Establishing Standards to Evaluate the Impact of Integrating Digital Health into Health Systems

Abstract

The key milestones in the rise of digital health illustrate efforts to bridge gaps in the evidence base, a shifting focus to scale-up and sustainability, growing attention to the precise costing of these strategies, and an emergent implementation science agenda that better characterizes the ecosystem—the social, political, economic, legal, and ethical context that supports digital health implementation—necessary to take digital health approaches to scale.

Perceptions of factors influencing the introduction and adoption of electronic immunization registries in Tanzania and Zambia: a mixed methods study

“…A unique aspect of this study was the use of the MAPS Toolkit, which allowed for comprehensive evaluation of the major components needed to introduce and adopt the EIRs as part of the scale-up plan. The EIRs deployed in Tanzania and Zambia were planned for scale prior to piloting the systems; this is a rare occurrence in LMICs where limited numbers of DHIs are scaled following a pilot or "proof-of-concept" deployment [6,7,17]. The variability in the success of EIR adoption is seen by the range of use amongst districts across each country.…”

“…The EIR implementation challenges are not unique to immunization programs, our results closely reflect what has been found for other DHI. The enabling ecosystem, or external factors, can greatly influence the success of these types of interventions [6,18]. An unpublished study has used data from the EIRs in Tanzania and Zambia to assess determinants of use and found significant differences by type of facility, district, and facility power source.…”

“…Findings from this study help to provide needed context to understand the degree of sustained use, which in turn influences the resulting immunization coverage. Successfully scaling up of a DHI is contingent on the intervention demonstrating an impact on health outcomes, which continues to remain a challenge as there are few studies that assess impact on clinical outcomes in LMICs [6,17,24]. Researchers and policy makers have recognized that an implementation science agenda is needed to help expand the breadth of relevant evidence available for DHIs, this includes conducting more mixed methods and quasi-experimental studies that can help understand the influence of contextual factors on implementation of an intervention [26].…”

“…However, despite the increased availability of technology, in LMICs there hasn't been a corresponding increase in DHIs taken to regional or national scale. Projects may suffer from "pilotitis" and simply have been deployed to demonstrate feasibility or were not built to sustain increasing client volumes and do not have the resources to scale and become institutionalized [6,7]. These interventions are complex; they require strong coordination and support across different health system levels and government departments and need significant capacities in technology and information to be properly implemented [8,9].…”

The complexity of scaling up an mHealth intervention. SMS for Life: A Case Study from a Health Systems Integration Perspective

“…A change in aim would have encouraged not only the reporting but also the action necessary to solve stockouts by the health system. The literature describes this output-driven set-up as a common shortfall of digital health projects being implemented around 2010 (23). In 2011, and as a response of this, the Bellagio Statement on eHealth Evidence was written, calling for better evaluation and research of digital health programmes (24).…”