Developing and Evaluating Digital Interventions to Promote Behavior Change in Health and Health Care: Recommendations Resulting From an International Workshop

Abstract

Devices and programs using digital technology to foster or support behavior change (digital interventions) are increasingly ubiquitous, being adopted for use in patient diagnosis and treatment, self-management of chronic diseases, and in primary prevention. They have been heralded as potentially revolutionizing the ways in which individuals can monitor and improve their health behaviors and health care by improving outcomes, reducing costs, and improving the patient experience. However, we are still mainly in the age of promise rather than delivery. Developing and evaluating these digital interventions presents new challenges and new versions of old challenges that require use of improved and perhaps entirely new methods for research and evaluation. This article discusses these challenges and provides recommendations aimed at accelerating the rate of progress in digital behavior intervention research and practice. Areas addressed include intervention development in a rapidly changing technological landscape, promoting user engagement, advancing the underpinning science and theory, evaluating effectiveness and cost-effectiveness, and addressing issues of regulatory, ethical, and information governance. This article is the result of a two-day international workshop on how to create, evaluate, and implement effective digital interventions in relation to health behaviors. It was held in London in September 2015 and was supported by the United Kingdom’s Medical Research Council (MRC), the National Institute for Health Research (NIHR), the Methodology Research Programme (PI Susan Michie), and the Robert Wood Johnson Foundation of the United States (PI Kevin Patrick). Important recommendations to manage the rapid pace of change include considering using emerging techniques from data science, machine learning, and Bayesian approaches and learning from other disciplines including computer science and engineering. With regard to assessing and promoting engagement, a key conclusion was that sustained engagement is not always required and that for each intervention it is useful to establish what constitutes “effective engagement,” that is, sufficient engagement to achieve the intended outcomes. The potential of digital interventions for testing and advancing theories of behavior change by generating ecologically valid, real-time objective data was recognized. Evaluations should include all phases of the development cycle, designed for generalizability, and consider new experimental designs to make the best use of rich data streams. Future health economics analyses need to recognize and model the complex and potentially far-reaching costs and benefits of digital interventions. In terms of governance, developers of digital behavior interventions should comply with existing regulatory frameworks, but with consideration for emerging standards around information governance, ethics, and interoperability.

Digital Characteristics and Dissemination Indicators to Optimize Delivery of Internet-Supported Mindfulness-Based Interventions for People With a Chronic Condition: Systematic Review (Preprint)

“…It is also unclear whether participants correctly understood the concept of mindfulness. However, it is also important to note that behaviour change prompted by digitally supported interventions may not require sustained or in-depth engagement with the program, as some users may only require a short period of intense engagement to initiate a habit or learn a skill, while others may need longer time periods and a more personalised approach [57]. This may partly explain the incongruity between low adherence rates and improved outcome measures observed in this review (e.g.…”

Digital Technology Disorder: Justification and a Proposed Model of Treatment

“…Since integrating aspects of sensitive design into systems can ensure that interactions with DTs are purely productive and cause minimal harm [63], DT-based treatments have the considerable potential of -improving outcomes, reducing costs, and improving the patient experience'' [53]. Using smartphones to deliver treatment aspects identified in the proposed model can enable several of the barriers to existing treatments to be removed.…”

A Framework for Analyzing and Measuring Usage and Engagement Data (AMUsED) in Digital Interventions: Viewpoint (Preprint)

“…However, to further our understanding of engagement, this method is best utilised as part of a multi-dimensional, mixed methods approach incorporating users' insights and experiences whilst using an intervention and away from it [6,8,15].…”

“…how an intervention worked, when and for whom [6,7]. Recent interest in 'effective engagement' considers these individual patterns of usage and the minimum level of engagement necessary for changes in target behavior to occur, including variation across individuals [6,[8][9][10]. Effective engagement may be achieved through using the intervention a certain amount.…”

“…However, efficient management and systematic analysis of large data sets can be challenging in the absence of detailed guiding frameworks [8], particularly if specific testable hypotheses/questions about usage have not been considered in advance of data collection. Existing guidelines encourage precise and standardized reporting for general analyses of digital interventions [19,20].…”