A design and evaluation framework for digital health interventions

Kowatsch, Tobias; Otto, Lena; Harperink, Samira; Cotti, Amanda; Schlieter, Hannes

doi:10.1515/itit-2019-0019

Cited by 110 publications

(109 citation statements)

References 75 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Evaluation measures were divided into 3 main types: technical performance, user experience, and health research measures. The first attempts toward evaluation frameworks for digital health interventions [ 82 ] and health chatbots [ 83 , 84 ] have been recently published. Depending on the facet under consideration, different metrics can be used: system performance and effectiveness can be evaluated using different computational metrics (eg, usability, ease of use, usefulness) [ 85 ].…”

Section: Discussionmentioning

confidence: 99%

Perceptions and Opinions of Patients About Mental Health Chatbots: Scoping Review

Abd‐Alrazaq¹,

Alajlani²,

Ali³

et al. 2021

J Med Internet Res

185

View full text Add to dashboard Cite

Background Chatbots have been used in the last decade to improve access to mental health care services. Perceptions and opinions of patients influence the adoption of chatbots for health care. Many studies have been conducted to assess the perceptions and opinions of patients about mental health chatbots. To the best of our knowledge, there has been no review of the evidence surrounding perceptions and opinions of patients about mental health chatbots. Objective This study aims to conduct a scoping review of the perceptions and opinions of patients about chatbots for mental health. Methods The scoping review was carried out in line with the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) extension for scoping reviews guidelines. Studies were identified by searching 8 electronic databases (eg, MEDLINE and Embase) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. In total, 2 reviewers independently selected studies and extracted data from the included studies. Data were synthesized using thematic analysis. Results Of 1072 citations retrieved, 37 unique studies were included in the review. The thematic analysis generated 10 themes from the findings of the studies: usefulness, ease of use, responsiveness, understandability, acceptability, attractiveness, trustworthiness, enjoyability, content, and comparisons. Conclusions The results demonstrated overall positive perceptions and opinions of patients about chatbots for mental health. Important issues to be addressed in the future are the linguistic capabilities of the chatbots: they have to be able to deal adequately with unexpected user input, provide high-quality responses, and have to show high variability in responses. To be useful for clinical practice, we have to find ways to harmonize chatbot content with individual treatment recommendations, that is, a personalization of chatbot conversations is required.

show abstract

Section: Discussionmentioning

confidence: 99%

Perceptions and Opinions of Patients About Mental Health Chatbots: Scoping Review

Abd‐Alrazaq¹,

Alajlani²,

Ali³

et al. 2021

J Med Internet Res

185

View full text Add to dashboard Cite

show abstract

“…Literature highlighted that the essence of data extraction is to record characteristics of the included studies and key information relevant to the review questions. 24 Congruent with the purpose and questions of this scoping review and taking reference from the existing digital health interventions design and evaluation framework, 34 we identified a priori categories and related variables including 'general information categories', 'key conceptual categories' and 'additional categories' with related variables described below as the data extraction framework to guide the extraction and charting of data from the included studies. A data chart will be developed based on the data extraction framework by our research team.…”

Section: Extracting and Charting The Results (Stage 4)mentioning

confidence: 99%

“…We drew on digital health intervention design and evaluation framework 34 to identify the following sets of variables which will be extracted from the included studies to strengthen the technological foundation of evidence: eHealth literacy, ease of use, perceived benefit, content quality, personalisation, adherence, safety, privacy and security.…”

Section: Additional Categoriesmentioning

confidence: 99%

Protocol for a scoping review of digital health for older adults with cancer and their families

et al. 2020

View full text Add to dashboard Cite

IntroductionThe potential for digital medicine and healthcare in geriatric oncology settings has received much attention. This scoping review will summarise the nature and extent of the existing literature that describes and examines digital health development, implementation, evaluation, outcome and experience for older adults with cancer, their families and their healthcare providers.Methods and analysisArksey and O'Malley’s six stages of scoping review methodology framework will be used. Searches will be conducted in Cochrane Central Register of Controlled Trials (CENTRAL), PubMed, Embase via OvidSP, Cumulative Index to Nursing and Allied Health Literature (CINAHL) Plus via EBSCO, Scopus and PsycINFO via OvidSP for published articles in peer-reviewed scientific journals from year 2000 onwards. In addition, we will screen databases for all prospectively registered trials. Research articles using quantitative or qualitative study design or reviews will be included if they describe or report the design, development or usability of digital health interventions in the treatment and care of patients 65 years of age or older with cancer and their families before, during and after cancer treatment. Grey literature will not be searched and included. Two investigators will independently perform the literature search, eligibility assessments and study selection. A Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram for the scoping reviews (PRISMA-ScR) will be used to delineate the search decision process. For included articles, the extracted results will be synthesised both quantitatively and qualitatively and reported under key conceptual categories of this scoping review. Research gaps and opportunities will be identified and summarised.Ethics and disseminationSince this review will only include published data, ethics approval will not be sought. The results of the review will be published in peer-reviewed scientific journals. We will also engage with relevant stakeholders within research team’s networks to determine suitable approaches for dissemination.

show abstract

“…The American Psychiatric Association (APA) App Evaluation framework offers a useful tool to guide informed decision making around apps. Supported by evidence [ 13 , 14 ], international stakeholders [ 15 ], and frequently cited in research [ 16 , 17 ] on app evaluation, the framework offers a simple and ethically grounded approach that first considers access, then in sequence privacy/safety, evidence, usability, and finally clinical integration. The hierarchical nature of the framework ensures consideration of factors frequently overlooked by many other app evaluation tools, such as privacy and safety [ 14 ].…”

Section: Methodsmentioning

confidence: 99%

Technology Evaluation and Assessment Criteria for Health Apps (TEACH-Apps): Pilot Study

et al. 2020

View full text Add to dashboard Cite

Background Despite the emergence of app evaluation tools, there remains no well-defined process receptive to diverse local needs, rigorous standards, and current content. The need for such a process to assist in the implementation of app evaluation across all medical fields is evident. Such a process has the potential to increase stakeholder engagement and catalyze interest and engagement with present-day app evaluation models. Objective This study aimed to develop and pilot test the Technology Evaluation and Assessment Criteria for Health apps (TEACH-apps). Methods Tailoring a well-known implementation framework, Replicating Effective Programs, we present a new process to approach the challenges faced in implementing app evaluation tools today. As a culmination of our experience implementing this process and feedback from stakeholders, we present the four-part process to aid the implementation of mobile health technology. This paper outlines the theory, evidence, and initial versions of the process. Results The TEACH-apps process is designed to be broadly usable and widely applicable across all fields of health. The process comprises four parts: (1) preconditions (eg, gathering apps and considering local needs), (2) preimplementation (eg, customizing criteria and offering digital skills training), (3) implementation (eg, evaluating apps and creating educational handouts), and (4) maintenance and evolution (eg, repeating the process every 90 days and updating content). TEACH-apps has been tested internally at our hospital, and there is growing interest in partnering health care facilities to test the system at their sites. Conclusions This implementation framework introduces a process that equips stakeholders, clinicians, and users with the foundational tools to make informed decisions around app use and increase app evaluation engagement. The application of this process may lead to the selection of more culturally appropriate and clinically relevant tools in health care.

show abstract

A design and evaluation framework for digital health interventions

Cited by 110 publications

References 75 publications

Perceptions and Opinions of Patients About Mental Health Chatbots: Scoping Review

Perceptions and Opinions of Patients About Mental Health Chatbots: Scoping Review

Protocol for a scoping review of digital health for older adults with cancer and their families

Technology Evaluation and Assessment Criteria for Health Apps (TEACH-Apps): Pilot Study

Contact Info

Product

Resources

About