A Chatbot Versus Physicians to Provide Information for Patients With Breast Cancer: Blind, Randomized Controlled Noninferiority Trial

Abstract

BackgroundThe data regarding the use of conversational agents in oncology are scarce.ObjectiveThe aim of this study was to verify whether an artificial conversational agent was able to provide answers to patients with breast cancer with a level of satisfaction similar to the answers given by a group of physicians.MethodsThis study is a blind, noninferiority randomized controlled trial that compared the information given by the chatbot, Vik, with that given by a multidisciplinary group of physicians to patients with breast cancer. Patients were women with breast cancer in treatment or in remission. The European Organisation for Research and Treatment of Cancer Quality of Life Group information questionnaire (EORTC QLQ-INFO25) was adapted and used to compare the quality of the information provided to patients by the physician or the chatbot. The primary outcome was to show that the answers given by the Vik chatbot to common questions asked by patients with breast cancer about their therapy management are at least as satisfying as answers given by a multidisciplinary medical committee by comparing the success rate in each group (defined by a score above 3). The secondary objective was to compare the average scores obtained by the chatbot and physicians for each INFO25 item.ResultsA total of 142 patients were included and randomized into two groups of 71. They were all female with a mean age of 42 years (SD 19). The success rates (as defined by a score >3) was 69% (49/71) in the chatbot group versus 64% (46/71) in the physicians group. The binomial test showed the noninferiority (P<.001) of the chatbot’s answers.ConclusionsThis is the first study that assessed an artificial conversational agent used to inform patients with cancer. The EORTC INFO25 scores from the chatbot were found to be noninferior to the scores of the physicians. Artificial conversational agents may save patients with minor health concerns from a visit to the doctor. This could allow clinicians to spend more time to treat patients who need a consultation the most.Trial RegistrationClinicaltrials.gov NCT03556813, https://tinyurl.com/rgtlehq

Clinical Trials for Artificial Intelligence in Cancer Diagnosis: A Cross-Sectional Study of Registered Trials in ClinicalTrials.gov

“…Two groups of randomly assigned patients asked 12 predefined questions that were previously answered by a chatbot or a medical committee and received the response from either a chatbot or a physician. The chatbot group had higher success rates (69 vs. 64%) than the physician group, which showed noninferiority (P < 0.001) (17).…”

Vik: A Chatbot to Support Patients with Chronic Diseases

“…In this phase III, non-inferiority, randomized, blind, controlled trial, the EORTC INFO25 scores from the chatbot were found to be non-inferior to the scores of the group of physicians [10]. As a consequence, the tools have the same limitation.…”

Association between patient‐initiated emails and overall 2‐year survival in cancer patients undergoing chemotherapy: Evidence from the real‐world setting

“…Commercial products using artificial intelligence to capture patient's symptoms and health concerns from emails to guide scheduling, and online tools that use chatbots to triage patient requests for appointments are already under development. 21,22 The interpretation of this study's results has limitations. First, we used email as a proxy for patient engagement, which might not be an appropriate assumption.…”