Background Digital health interventions (DHIs) have the potential to improve public health by combining effective interventions and population reach. However, what biomedical researchers and digital developers consider an effective intervention differs, thereby creating an ongoing challenge to integrating their respective approaches when evaluating DHIs. Objective This study aims to report on the Public Health England (PHE) initiative set out to operationalize an evaluation framework that combines biomedical and digital approaches and demonstrates the impact, cost-effectiveness, and benefit of DHIs on public health. Methods We comprised a multidisciplinary project team including service designers, academics, and public health professionals and used user-centered design methods, such as qualitative research, engagement with end users and stakeholders, and iterative learning. The iterative approach enabled the team to sequentially define the problem, understand user needs, identify opportunity areas, develop concepts, test prototypes, and plan service implementation. Stakeholders, senior leaders from PHE, and a working group critiqued the outputs. Results We identified 26 themes and 82 user needs from semistructured interviews (N=15), expressed as 46 Jobs To Be Done, which were then validated across the journey of evaluation design for a DHI. We identified seven essential concepts for evaluating DHIs: evaluation thinking, evaluation canvas, contract assistant, testing toolkit, development history, data hub, and publish health outcomes. Of these, three concepts were prioritized for further testing and development, and subsequently refined into the proposed PHE Evaluation Service for public health DHIs. Testing with PHE’s Couch-to-5K app digital team confirmed the viability, desirability, and feasibility of both the evaluation approach and the Evaluation Service. Conclusions An iterative, user-centered design approach enabled PHE to combine the strengths of academic and biomedical disciplines with the expertise of nonacademic and digital developers for evaluating DHIs. Design-led methodologies can add value to public health settings. The subsequent service, now known as Evaluating Digital Health Products, is currently in use by health bodies in the United Kingdom and is available to others for tackling the problem of evaluating DHIs pragmatically and responsively.
BACKGROUND Digital health interventions have potential to improve public health by combining effective intervention and population reach. However, what biomedical researchers and digital developers consider an effective intervention differs, thereby creating an ongoing challenge to integrate their respective approaches when evaluating digital health interventions. OBJECTIVE Public Health England set out to operationalise an evaluation framework that combines biomedical and digital approaches, and demonstrates the impact, cost-effectiveness and benefit of digital health interventions to public health. METHODS A multidisciplinary project team, composed of service designers, academics and public health professionals, employed user-centred design methods such as qualitative research, engagement with end-users and stakeholders, and iterative learning. An iterative approach enabled the team to sequentially define the problem, understand user needs, identify opportunity areas, develop concepts, test prototypes, and plan service implementation. Outputs were critiqued by stakeholders, system leaders and a Working Group. RESULTS Semi-structured interviews (N=15) identified 26 themes and 82 user needs, expressed as 46 Jobs To Be Done, which were then validated across the journey of evaluation design for a digital health intervention. Seven essential concepts for evaluating digital health interventions were identified: (i) Evaluation Thinking; (ii) Evaluation Canvas; (iii) Contract Assistant; (iv) Testing Toolkit; (v) Development History; (vi) Data Hub, and (vii) Publish Health Outcomes. Three concepts were prioritised for further testing and development and subsequently refined into the proposed Public Health England Evaluation Service for public health digital health interventions. Testing with Public Health England's Couch-to-5K App digital team confirmed the viability, desirability and feasibility of both the evaluation approach and the Evaluation Service. CONCLUSIONS An iterative, user-centred design approach enabled Public Health England to combine the strengths of academic and biomedical disciplines alongside the expertise of non-academic and/or digital developers for evaluating digital health interventions. Design-led methodologies can add value in a public health setting. The subsequent service, now known as Evaluating Digital Health Products, is currently in use by health bodies in the United Kingdom and available to others tackling the problem of evaluating digital health interventions pragmatically and responsively.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.