Reliability of Commercial Voice Assistants’ Responses to Health-Related Questions in Noncommunicable Disease Management: Factorial Experiment Assessing Response Rate and Source of Information

Bérubé, Caterina; Kovacs, Zsolt Ferenc; Fleisch, Elgar; Kowatsch, Tobias

doi:10.2196/32161

Cited by 6 publications

(4 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to some previous work [ 9 ], we preemptively decided to include studies that used a “Wizard of Oz” protocol, in which participants interact with what they believe to be an autonomous AI system but is actually an interface being controlled by a concealed human operator (the “wizard”). “Wizard of Oz” experiments are often used in the early phases of system design and testing to address design and usability issues before time and resources are invested in software development [ 10 , 13 ]. While the simulated conversational agents used in the “Wizard of Oz” experiments are not themselves autonomous AI systems, we nonetheless deemed them relevant since they contain data on how users perceive and interact with vaccine communications delivered by (what they perceive to be) autonomous AI systems.…”

Section: Methodsmentioning

confidence: 99%

“…Our review contributes to the emerging literature on conversational AI in the health field [1,4,[8][9][10][11]. Previous related work includes a scoping review [3] and 2 systematic reviews [10,11] of conversational AI within the health field as a whole; a number of domain-or disease-specific reviews (eg, chatbots focused on noncommunicable diseases, COVID-19, sexual health, and smoking cessation) [1,9,12,13]; and some technology-specific studies (eg, health information provided by voice assistants such as Siri and Alexa or via smartphone apps) [8,9,12].…”

Section: Introductionmentioning

confidence: 99%

“…Focused on vaccine-related chatbots In contrast to some previous work [9], we preemptively decided to include studies that used a "Wizard of Oz" protocol, in which participants interact with what they believe to be an autonomous AI system but is actually an interface being controlled by a concealed human operator (the "wizard"). "Wizard of Oz" experiments are often used in the early phases of system design and testing to address design and usability issues before time and resources are invested in software development [10,13].…”

mentioning

confidence: 99%

See 2 more Smart Citations

Conversational AI and Vaccine Communication: Systematic Review of the Evidence

Passanante¹,

Pertwee²,

Lin³

et al. 2023

J Med Internet Res

View full text Add to dashboard Cite

Background Since the mid-2010s, use of conversational artificial intelligence (AI; chatbots) in health care has expanded significantly, especially in the context of increased burdens on health systems and restrictions on in-person consultations with health care providers during the COVID-19 pandemic. One emerging use for conversational AI is to capture evolving questions and communicate information about vaccines and vaccination. Objective The objective of this systematic review was to examine documented uses and evidence on the effectiveness of conversational AI for vaccine communication. Methods This systematic review was conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. PubMed, Web of Science, PsycINFO, MEDLINE, Scopus, CINAHL Complete, Cochrane Library, Embase, Epistemonikos, Global Health, Global Index Medicus, Academic Search Complete, and the University of London library database were searched for papers on the use of conversational AI for vaccine communication. The inclusion criteria were studies that included (1) documented instances of conversational AI being used for the purpose of vaccine communication and (2) evaluation data on the impact and effectiveness of the intervention. Results After duplicates were removed, the review identified 496 unique records, which were then screened by title and abstract, of which 38 were identified for full-text review. Seven fit the inclusion criteria and were assessed and summarized in the findings of this review. Overall, vaccine chatbots deployed to date have been relatively simple in their design and have mainly been used to provide factual information to users in response to their questions about vaccines. Additionally, chatbots have been used for vaccination scheduling, appointment reminders, debunking misinformation, and, in some cases, for vaccine counseling and persuasion. Available evidence suggests that chatbots can have a positive effect on vaccine attitudes; however, studies were typically exploratory in nature, and some lacked a control group or had very small sample sizes. Conclusions The review found evidence of potential benefits from conversational AI for vaccine communication. Factors that may contribute to the effectiveness of vaccine chatbots include their ability to provide credible and personalized information in real time, the familiarity and accessibility of the chatbot platform, and the extent to which interactions with the chatbot feel “natural” to users. However, evaluations have focused on the short-term, direct effects of chatbots on their users. The potential longer-term and societal impacts of conversational AI have yet to be analyzed. In addition, existing studies do not adequately address how ethics apply in the field of conversational AI around vaccines. In a context where further digitalization of vaccine communication can be anticipated, additional high-quality research will be required across all these areas.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Conversational AI and Vaccine Communication: Systematic Review of the Evidence

Passanante¹,

Pertwee²,

Lin³

et al. 2023

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…CAs have been variously used and studied in health care for supporting behavioral health and healthy living [3,4]; health information seeking [5][6][7][8][9]; appointment, medication, symptom tracking and chronic condition management [10][11][12]; and facilitating COVID-19 screening and information sharing [13,14]. Mobile phone ownership enables and increases the potential applications, availability, and access to CAs in practice.…”

Section: Introductionmentioning

confidence: 99%

Design and Evaluation Challenges of Conversational Agents in Health Care and Well-being: Selective Review Study

Kocaballı¹,

Sezgın²,

Clark³

et al. 2022

J Med Internet Res

View full text Add to dashboard Cite

Background Health care and well-being are 2 main interconnected application areas of conversational agents (CAs). There is a significant increase in research, development, and commercial implementations in this area. In parallel to the increasing interest, new challenges in designing and evaluating CAs have emerged. Objective This study aims to identify key design, development, and evaluation challenges of CAs in health care and well-being research. The focus is on the very recent projects with their emerging challenges. Methods A review study was conducted with 17 invited studies, most of which were presented at the ACM (Association for Computing Machinery) CHI 2020 conference workshop on CAs for health and well-being. Eligibility criteria required the studies to involve a CA applied to a health or well-being project (ongoing or recently finished). The participating studies were asked to report on their projects’ design and evaluation challenges. We used thematic analysis to review the studies. Results The findings include a range of topics from primary care to caring for older adults to health coaching. We identified 4 major themes: (1) Domain Information and Integration, (2) User-System Interaction and Partnership, (3) Evaluation, and (4) Conversational Competence. Conclusions CAs proved their worth during the pandemic as health screening tools, and are expected to stay to further support various health care domains, especially personal health care. Growth in investment in CAs also shows the value as a personal assistant. Our study shows that while some challenges are shared with other CA application areas, safety and privacy remain the major challenges in the health care and well-being domains. An increased level of collaboration across different institutions and entities may be a promising direction to address some of the major challenges that otherwise would be too complex to be addressed by the projects with their limited scope and budget.

show abstract