GPT-4 for triaging ophthalmic symptoms

Waisberg, Ethan; Ong, Joshua; Zaman, Nasif; Kamran, Sharif Amit; Sarker, Prithul; Tavakkoli, Alireza; Lee, Andrew G.

doi:10.1038/s41433-023-02595-9

Cited by 21 publications

(9 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The analysis underlines the fact that each AI model, including ChatGPT, GPT-4, Bard, and Gemini AI, possesses unique strengths and weaknesses, making them suitable for different applications and use cases. It is important to note that further advancements are necessary prior to being the use of AI chatbots in clinical settings [ 12 , 13 ].…”

Section: Discussionmentioning

confidence: 99%

Google DeepMind’s gemini AI versus ChatGPT: a comparative analysis in ophthalmology

Masalkhi,

Ong,

Waisberg

et al. 2024

Eye

Self Cite

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

Google DeepMind’s gemini AI versus ChatGPT: a comparative analysis in ophthalmology

Masalkhi,

Ong,

Waisberg

et al. 2024

Eye

Self Cite

View full text Add to dashboard Cite

“…Moreover, there remain controversial ethical questions about what roles should and should not be assigned to inanimate AI models, and to what extent human clinicians must remain responsible for their patients [ 3 ]. However, the remarkable performance of GPT-4 in ophthalmology examination questions suggests that LLMs may be able to provide useful input in clinical contexts, either to assist clinicians in their day-to-day work or with their education or preparation for examinations [ 3 , 13 , 14 , 27 ]. Further improvement in performance may be obtained by specific fine-tuning of models with high quality ophthalmological text data, requiring curation and deidentification [ 29 ].…”

Section: Discussionmentioning

confidence: 99%

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Thirunavukarasu,

Mahmood,

Malem

et al. 2024

PLOS Digit Health

View full text Add to dashboard Cite

Large language models (LLMs) underlie remarkable recent advanced in natural language processing, and they are beginning to be applied in clinical contexts. We aimed to evaluate the clinical potential of state-of-the-art LLMs in ophthalmology using a more robust benchmark than raw examination scores. We trialled GPT-3.5 and GPT-4 on 347 ophthalmology questions before GPT-3.5, GPT-4, PaLM 2, LLaMA, expert ophthalmologists, and doctors in training were trialled on a mock examination of 87 questions. Performance was analysed with respect to question subject and type (first order recall and higher order reasoning). Masked ophthalmologists graded the accuracy, relevance, and overall preference of GPT-3.5 and GPT-4 responses to the same questions. The performance of GPT-4 (69%) was superior to GPT-3.5 (48%), LLaMA (32%), and PaLM 2 (56%). GPT-4 compared favourably with expert ophthalmologists (median 76%, range 64–90%), ophthalmology trainees (median 59%, range 57–63%), and unspecialised junior doctors (median 43%, range 41–44%). Low agreement between LLMs and doctors reflected idiosyncratic differences in knowledge and reasoning with overall consistency across subjects and types (p>0.05). All ophthalmologists preferred GPT-4 responses over GPT-3.5 and rated the accuracy and relevance of GPT-4 as higher (p<0.05). LLMs are approaching expert-level knowledge and reasoning skills in ophthalmology. In view of the comparable or superior performance to trainee-grade ophthalmologists and unspecialised junior doctors, state-of-the-art LLMs such as GPT-4 may provide useful medical advice and assistance where access to expert ophthalmologists is limited. Clinical benchmarks provide useful assays of LLM capabilities in healthcare before clinical trials can be designed and conducted.

show abstract

“…5 Unfortunately, both of the cited references 9,10 regarding the significance of AI models in the early detection and management of myopia are out of scope and lack merit, as one is an editorial on general ophthalmology and the other concerns the retina; neither mentions myopia. Similarly, the other references 11,12 are an abstract on ChatGPT-3's performance in answering ophthalmology cases and correspondence regarding ChatGPT-4's triaging ability; both lack perspective or context and do not corroborate what Ray is describing in the letter. The author then speculates about the use of AI and LLMs without published data or any scientific grounds.…”

Section: Authors' Reply: Assessing the Utility Of Chatgpt As An Artif...mentioning

confidence: 99%

Authors' Reply: Assessing the utility of ChatGPT as an artificial intelligence‐based large language model for information to answer questions on myopia

Biswas,

Logan,

Davies

et al. 2023

Ophthalmic Physiologic Optic

View full text Add to dashboard Cite

GPT-4 for triaging ophthalmic symptoms

Cited by 21 publications

References 3 publications

Google DeepMind’s gemini AI versus ChatGPT: a comparative analysis in ophthalmology

Google DeepMind’s gemini AI versus ChatGPT: a comparative analysis in ophthalmology

Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study

Authors' Reply: Assessing the utility of ChatGPT as an artificial intelligence‐based large language model for information to answer questions on myopia

Contact Info

Product

Resources

About