Chatbots have become an increasingly popular tool in the field of health services and communications. Despite chatbots’ significance amid the COVID-19 pandemic, few studies have performed a rigorous evaluation of the effectiveness of chatbots in improving vaccine confidence and acceptance. In Thailand, Hong Kong, and Singapore, from February 11th to June 30th, 2022, we conducted multisite randomised controlled trials (RCT) on 2,045 adult guardians of children and seniors who were unvaccinated or had delayed vaccinations. After a week of using COVID-19 vaccine chatbots, the differences in vaccine confidence and acceptance were compared between the intervention and control groups. Compared to non-users, fewer chatbot users reported decreased confidence in vaccine effectiveness in the Thailand child group [Intervention: 4.3 % vs. Control: 17%, P = 0.023]. However, more chatbot users reported decreased vaccine acceptance [26% vs. 12%, P = 0.028] in Hong Kong child group and decreased vaccine confidence in safety [29% vs. 10%, P = 0.041] in Singapore child group. There was no statistically significant change in vaccine confidence or acceptance in the Hong Kong senior group. Employing the RE-AIM framework, process evaluation indicated strong acceptance and implementation support for vaccine chatbots from stakeholders, with high levels of sustainability and scalability. This multisite, parallel RCT study on vaccine chatbots found mixed success in improving vaccine confidence and acceptance among unvaccinated Asian subpopulations. Further studies that link chatbot usage and real-world vaccine uptake are needed to augment evidence for employing vaccine chatbots to advance vaccine confidence and acceptance.

Background Since the mid-2010s, use of conversational artificial intelligence (AI; chatbots) in health care has expanded significantly, especially in the context of increased burdens on health systems and restrictions on in-person consultations with health care providers during the COVID-19 pandemic. One emerging use for conversational AI is to capture evolving questions and communicate information about vaccines and vaccination. Objective The objective of this systematic review was to examine documented uses and evidence on the effectiveness of conversational AI for vaccine communication. Methods This systematic review was conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. PubMed, Web of Science, PsycINFO, MEDLINE, Scopus, CINAHL Complete, Cochrane Library, Embase, Epistemonikos, Global Health, Global Index Medicus, Academic Search Complete, and the University of London library database were searched for papers on the use of conversational AI for vaccine communication. The inclusion criteria were studies that included (1) documented instances of conversational AI being used for the purpose of vaccine communication and (2) evaluation data on the impact and effectiveness of the intervention. Results After duplicates were removed, the review identified 496 unique records, which were then screened by title and abstract, of which 38 were identified for full-text review. Seven fit the inclusion criteria and were assessed and summarized in the findings of this review. Overall, vaccine chatbots deployed to date have been relatively simple in their design and have mainly been used to provide factual information to users in response to their questions about vaccines. Additionally, chatbots have been used for vaccination scheduling, appointment reminders, debunking misinformation, and, in some cases, for vaccine counseling and persuasion. Available evidence suggests that chatbots can have a positive effect on vaccine attitudes; however, studies were typically exploratory in nature, and some lacked a control group or had very small sample sizes. Conclusions The review found evidence of potential benefits from conversational AI for vaccine communication. Factors that may contribute to the effectiveness of vaccine chatbots include their ability to provide credible and personalized information in real time, the familiarity and accessibility of the chatbot platform, and the extent to which interactions with the chatbot feel “natural” to users. However, evaluations have focused on the short-term, direct effects of chatbots on their users. The potential longer-term and societal impacts of conversational AI have yet to be analyzed. In addition, existing studies do not adequately address how ethics apply in the field of conversational AI around vaccines. In a context where further digitalization of vaccine communication can be anticipated, additional high-quality research will be required across all these areas.

Background Little is known about how social media platforms can be used to increase COVID-19 vaccine intent. We aimed to investigate the effect of social media-based interventions on vaccine hesitancy in Japan. Methods We conducted a three-arm randomized controlled trial between 5 November 2021 and 9 January 2022. Japanese aged 20 or above who had not received any COVID-19 vaccine and did not intend to be vaccinated were randomly assigned to one of the following three groups: (i) a control group (with no intervention), (ii) a group with a free chatbot in a popular messenger app called ‘LINE,’ which provided general information on COVID-19 vaccines and (iii) a group with free webinars where healthcare professionals interactively provided participants with the information on COVID-19 vaccines. The vaccine intention (VI) and three pre-defined Vaccine Confidence Index (VCI), including the importance, safety, and effectiveness of COVID-19 vaccines, were compared. Results 1,158 persons were included, and the baseline characteristics and demographics were balanced across three groups (Table 1). Among 386 persons assigned to the chatbot group, 231 (59.8%) accessed the chatbot and answered the post-survey. The post-survey revealed no significant difference in VI or VCI between the chatbot group and the control group (Table 2). Among 386 persons assigned to the webinar group, 207 (53.6%) attended webinars and answered the post-survey. The post-survey revealed no difference in VI between the webinar group and the control group. However, the VCI for the importance and the effectiveness significantly increased in the webinar group. There was no difference in VCI for the safety. VCI for the importance and the effectiveness in the control group decreased without any intervention during the study period. Vaccine intention and confidence after interventions Table 1Table 2 Conclusion While this study demonstrated that neither the chatbot nor the webinar changed VI, VCI for the importance and the effectiveness significantly increased with the webinar intervention. Interactive webinars with live Q and A provided by professionals may have a role in increasing COVID-19 vaccine confidence. Given the degree of vaccine hesitancy worsened over time in the control group, timely intervention is required. Disclosures All Authors: No reported disclosures.

IntroductionWe investigated the effect of social media-based interventions on COVID-19 vaccine intention (VI) and confidence in Japan.MethodsWe conducted a three-arm randomised controlled trial between 5 November 2021 and 9 January 2022 during a low incidence (<1000/day) of COVID-19 in Japan in the midst of the second and the third waves. Japanese citizens aged ≥20 who had not received any COVID-19 vaccine and did not intend to be vaccinated were randomly assigned to one of the following three groups: (1) a control group, (2) a group using a mobile app chatbot providing information on COVID-19 vaccines and (3) a group using interactive webinars with health professionals. VI and predefined Vaccine Confidence Index (VCI) measuring confidence in the importance, safety and effectiveness were compared before and after the interventions under intention-to-treat principle. Logistic regression models were used to investigate the effect of each intervention on postintervention VI and changes of VCI compared with control.ResultsAmong 386 participants in each group, 359 (93.0%), 231 (59.8%) and 207 (53.6%) completed the postsurvey for the control, chatbot and webinar groups, respectively. The average duration between the intervention and the postsurvey was 32 days in chatbot group and 27 days in webinar group. VI increased from 0% to 18.5% (95% CI 14.5%, 22.5%) in control group, 15.4% (95% CI 10.8%, 20.1%) in chatbot group and 19.7% (95% CI 14.5%, 24.9%) in webinar group without significant difference (OR for improvement=0.8 (95% CI 0.5, 1.3), p=0.33 between chatbot and control, OR=1.1 (95% CI 0.7, 1.6), p=0.73 between webinar and control). VCI change tended to be larger in chatbot group compared with control group without significant difference (3.3% vs −2.5% in importance, OR for improvement=1.3 (95% CI 0.9, 2.0), p=0.18; 2.5% vs 1.9% in safety, OR=1.1 (95% CI 0.7, 1.9), p=0.62; −2.4% vs −7.6% in effectiveness, OR=1.4 (95% CI 0.9, 2.1), p=0.09). Improvement in VCI was larger in webinar group compared with control group for importance (7.8% vs −2.5%, OR=1.8 (95% CI 1.2, 2.8), p<0.01), effectiveness (6.4% vs −7.6%, OR=2.2 (95% CI 1.4, 3.4), p<0.01) and safety (6.0% vs 1.9%, OR=1.6 (95% CI 1.0, 2.6), p=0.08).ConclusionThis study demonstrated that neither the chatbot nor the webinar changed VI importantly compared with control. Interactive webinars could be an effective tool to change vaccine confidence. Further study is needed to identify risk factors associated with decreased vaccine confidence and investigate what intervention can increase VI and vaccine confidence for COVID-19 vaccines.Trial registration numberUMIN000045747.

During the COVID-19 pandemic, the emergence of the infodemic and vaccine hesitancy posed a significant challenge to adequate vaccine uptake. In response, conversational AI services such as chatbots have become an increasingly popular tool in the field of health service delivery and communication to increase individuals’ health literacy and vaccination intention. However, few studies have performed a rigorous evaluation of the effectiveness of chatbots as a means of improving vaccine confidence and acceptance. In Thailand, Hong Kong, and Singapore, from February 11th to June 30th, 2022, we conducted multisite randomised controlled trials (RCT) on 2,045 adults with unvaccinated dependent family members who were vulnerable (i.e., seniors) and had been refusing/delaying vaccination, or newly eligible for vaccines (i.e., children). After a week of using multilingual COVID-19 vaccine chatbots, the differences in vaccine confidence - measured by the Vaccine Confidence Index - and acceptance were compared between the intervention and control groups. Factors of vaccine confidence and acceptance were explored. Compared to non-users, a smaller proportion of chatbot users reported a decrease of confidence in vaccine effectiveness in the Thailand child group [Intervention: 4·3% vs. Control: 17%, P = 0·023] and Hong Kong child group [10% vs. 26%, P = 0·034], and of vaccine effectiveness in reducing severe conditions in the Thailand senior group [12% vs. 21%, P = 0·024]. There was no significant change in vaccine confidence or acceptance in the Singapore child group and Hong Kong senior group. Employing the RE-AIM framework, process evaluation indicated strong acceptance and implementation support for vaccine chatbots from stakeholders, with high levels of sustainability and scalability. This study was the first multisite, parallel RCT on vaccine chatbots and reported mixed success in improving vaccine confidence and acceptance among highly hesitant Asian subpopulations. Deploying chatbots as a complement to existing vaccination strategies could identify users’ main concerns for rejecting/delaying vaccination and facilitate a targeted communication and engagement strategy. Trial registration: NCT05424952