Evaluating ChatGPT's efficacy and readability to common pediatric ophthalmology and strabismus-related questions

Ahmed, H Shafeeq; Thrishulamurthy, Chinmayee J

doi:10.1177/11206721241272251

Cited by 2 publications

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Chatbots in healthcare: A study of readability and response accuracy in answers to questions about hypertension. (Preprint)

Olszewski,

Brzeziński,

Watros

et al. 2024

Preprint

View full text Add to dashboard Cite

BACKGROUND AI-powered chatbots, using Large Language Models, may effectively answer questions from patients with hypertension, providing responses that are accurate, empathetic, and easy to read. OBJECTIVE This study evaluates the performance of three such chatbots in delivering quality responses. METHODS One hundred questions were randomly selected from the Reddit forum r/hypertension and submitted to three publicly available chatbots (ChatGPT-3.5, Microsoft Copilot, Gemini), anonymized as A, B, and C. Two independent medical professionals assessed the accuracy and empathy of their responses using Likert scales. Additionally, 300 responses were analyzed with the WebFX readability tool to measure various readability indices. RESULTS In total, 300 responses were evaluated. Chatbot A generated the most extensive responses, with an average of 13 sentences per reply, while Chatbot B had the shortest replies. Chatbot C achieved the highest score on the Flesch Reading Ease Scale, indicating better readability, while Chatbot A scored the lowest. Other readability metrics, including the Flesch-Kincaid Grade Level, Gunning Fog Score, and others, also showed significant differences among the chatbots, reflecting variability in readability. CONCLUSIONS The study indicates that while all chatbots can produce professional responses, their readability varies significantly. These findings underscore the potential of AI chatbots in patient education. However, they also highlight the urgent need for further optimization to enhance the comprehensibility of their outputs.

show abstract

Chatbots in healthcare: A study of readability and response accuracy in answers to questions about hypertension. (Preprint)

Olszewski,

Brzeziński,

Watros

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

Opportunities and Challenges of Chatbots in Ophthalmology: A Narrative Review

Sabaner,

Anguita,

Antaki

et al. 2024

JPM

View full text Add to dashboard Cite

Artificial intelligence (AI) is becoming increasingly influential in ophthalmology, particularly through advancements in machine learning, deep learning, robotics, neural networks, and natural language processing (NLP). Among these, NLP-based chatbots are the most readily accessible and are driven by AI-based large language models (LLMs). These chatbots have facilitated new research avenues and have gained traction in both clinical and surgical applications in ophthalmology. They are also increasingly being utilized in studies on ophthalmology-related exams, particularly those containing multiple-choice questions (MCQs). This narrative review evaluates both the opportunities and the challenges of integrating chatbots into ophthalmology research, with separate assessments of studies involving open- and close-ended questions. While chatbots have demonstrated sufficient accuracy in handling MCQ-based studies, supporting their use in education, additional exam security measures are necessary. The research on open-ended question responses suggests that AI-based LLM chatbots could be applied across nearly all areas of ophthalmology. They have shown promise for addressing patient inquiries, offering medical advice, patient education, supporting triage, facilitating diagnosis and differential diagnosis, and aiding in surgical planning. However, the ethical implications, confidentiality concerns, physician liability, and issues surrounding patient privacy remain pressing challenges. Although AI has demonstrated significant promise in clinical patient care, it is currently most effective as a supportive tool rather than as a replacement for human physicians.

show abstract

Evaluating ChatGPT's efficacy and readability to common pediatric ophthalmology and strabismus-related questions

Cited by 2 publications

References 24 publications

Chatbots in healthcare: A study of readability and response accuracy in answers to questions about hypertension. (Preprint)

Chatbots in healthcare: A study of readability and response accuracy in answers to questions about hypertension. (Preprint)

Opportunities and Challenges of Chatbots in Ophthalmology: A Narrative Review

Contact Info

Product

Resources

About