Assessing the performances of a chatbot to collect real-life data of patients suffering from primary headache disorders

Abstract

Background There are many scales for screening the impact of a disease. These scales are generally used to diagnose or assess the type and severity of a disease and are carried out by doctors. The chatbot helps patients suffering from primary headache disorders through personalized text messages. It could be used to collect patient-reported outcomes. Objective The aims of this study were (1) to study whether the collection and analysis of remote scores, without prior medical intervention, are possible by a chatbot, (2) to perform suggested diagnosis and define the type of headaches, and (3) to assess the patient satisfaction and engagement with the chatbot. Method Voluntary users of the chatbot were recruited online. They had to be over 18 and have a personal history of headaches. A questionnaire was presented (1) by text messages to the participants to evaluate migraines (2) based on the criteria of the International Headache Society. Then, the Likert scale (3) was used to assess overall satisfaction with the use of the chatbot. Results We included 610 participants with primary headache disorders. A total of 89.94% (572/610) participants had fully completed the questionnaire (eight items), 4.72% (30/610) had partially completed it, and 5.41% (33) had refused to complete it. Statistical analysis was performed on 86.01% (547/610) of participants. Auto diagnostic showed that 14.26% (78/547) participants had a tension headache, and 85.74% (469/547) had a probable migraine. In this population, 15.78% (74/469) suffered from migraine without probable aura, and 84.22% (395/469) had migraine without aura. The patient’s age had a significant incidence regarding the auto diagnosis ( P = .008<.05). The evaluation of overall satisfaction shows that a total of 93.9% (599/610) of users were satisfied or very satisfied regarding the timeliness of responses the chatbot provides. Conclusion The study confirmed that it was possible to obtain such a collection remotely, and quickly (average time of 3.24 min) with a high success rate (89.67% (547/610) participants who had fully completed the IHS questionnaire). Users were strongly engaged through chatbot: out of the total number of participants, we observed a very low number of uncompleted questionnaires (6.23% (38/610)). Conversational agents can be used to remotely collect data on the nature of the symptoms of patients suffering from primary headache disorders. These results are promising regarding patient engagement and trust in the chatbot.

Multilingual Framework for Risk Assessment and Symptom Tracking (MRAST)

“…As already highlighted in the introduction, the main disadvantages of ePROMs from a patient’s perspective, and reasons for patients not using ePROMs, can be summarized as (i) ability to use (e.g., physical ability due to health issues), (ii) engagement (e.g., patients do not find them relevant because no symptoms exist), (iii) technical issues and usability (e.g., low technical proficiency), (iv) data security and trust [ 14 , 52 ]. Specifically tackling technical issues and usability, chatbots have been an efficient solution to improve usability and simplify the app functionalities and user experience [ 45 , 59 , 63 , 64 , 65 ]. Namely, chatbots exploit artificial intelligence and natural language processing to interact with patients without human intervention.…”

Overview of Chatbots with special emphasis on artificial intelligence-enabled ChatGPT in medical science

“…Presently, "expert" level on U.S. Medical Licensing Exam-style questions. (Chaix et al, 2022) 26. Chatbot This Chatbot predominantly applicable in health care, with the capability for complex dialog management and conversational flexibility in in medicine.…”