Application effect of an artificial intelligence-based fundus screening system: evaluation in a clinical setting and population screening

Cao, Shujuan; Zhang, Rong-Pei; Jiang, Aixin; Kuerban, Mayila; Wumaier, Aizezi; Wu, Jianhua; Xie, Kaihua; Aizezi, Mireayi; Tuersun, Abudurexiti; Liang, Xinghua; Chen, Rongxin

doi:10.1186/s12938-023-01097-9

Cited by 7 publications

(3 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with traditional computer-vision AI systems that can interpret visual data, LLMs are easily accessible and do not necessarily require prior familiarity with computer vision, AI, or coding . While our current investigation found that the chatbot achieved an accuracy of 65% when responding to multiple-choice questions requiring ophthalmic imaging interpretation, its performance remains inferior to previously published AI systems designed for screening or diagnosing retinal pathologies from ophthalmic imaging, such as OCT scans and fundus images . For instance, relative to professional graders, a previous deep learning system by Ting et al achieved a sensitivity of 90.5% and specificity of 91.6% for detecting referable diabetic retinopathy, a sensitivity of 100% and specificity of 91.1% for vision-threatening diabetic retinopathy, a sensitivity of 96.4% and specificity of 87.2% for possible glaucoma, and a sensitivity of 93.2% and specificity of 88.7% for age-related macular degeneration using a validation dataset of retinal images.…”

Section: Discussioncontrasting

confidence: 57%

“…17 While our current investigation found that the chatbot achieved an accuracy of 65% when responding to multiple-choice questions requiring ophthalmic imaging interpretation, its performance remains inferior to previously published AI systems designed for screening or diagnosing retinal pathologies from ophthalmic imaging, such as OCT scans and fundus images. [18][19][20] For instance, relative to professional graders, a previous deep learning system by Ting et al 21 achieved a sensitivity of 90.5% and specificity of 91.6% for detecting referable diabetic retinopathy, a sensitivity of 100% and specificity of 91.1% for vision-threatening diabetic retinopathy, a sensitivity of 96.4% and specificity of 87.2% for possible glaucoma, and a sensitivity of 93.2% and specificity of 88.7% for age-related macular degeneration using a validation dataset of retinal images. Overall, AI-assisted OCT analysis can be highly accurate, sensitive, and specific for the diagnosis of retinal disease, and it is possible that the incorporation of more robust AI algorithms into LLMs may further improve their multimodal capabilities.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Accuracy of an Artificial Intelligence Chatbot’s Interpretation of Clinical Ophthalmic Images

Mihalache,

Huang,

Popovic

et al. 2024

JAMA Ophthalmol

View full text Add to dashboard Cite

ImportanceOphthalmology is reliant on effective interpretation of multimodal imaging to ensure diagnostic accuracy. The new ability of ChatGPT-4 (OpenAI) to interpret ophthalmic images has not yet been explored.ObjectiveTo evaluate the performance of the novel release of an artificial intelligence chatbot that is capable of processing imaging data.Design, Setting, and ParticipantsThis cross-sectional study used a publicly available dataset of ophthalmic cases from OCTCases, a medical education platform based out of the Department of Ophthalmology and Vision Sciences at the University of Toronto, with accompanying clinical multimodal imaging and multiple-choice questions. Across 137 available cases, 136 contained multiple-choice questions (99%).ExposuresThe chatbot answered questions requiring multimodal input from October 16 to October 23, 2023.Main Outcomes and MeasuresThe primary outcome was the accuracy of the chatbot in answering multiple-choice questions pertaining to image recognition in ophthalmic cases, measured as the proportion of correct responses. χ2 Tests were conducted to compare the proportion of correct responses across different ophthalmic subspecialties.ResultsA total of 429 multiple-choice questions from 136 ophthalmic cases and 448 images were included in the analysis. The chatbot answered 299 of multiple-choice questions correctly across all cases (70%). The chatbot’s performance was better on retina questions than neuro-ophthalmology questions (77% vs 58%; difference = 18%; 95% CI, 7.5%-29.4%; χ21 = 11.4; P &lt; .001). The chatbot achieved a better performance on nonimage–based questions compared with image-based questions (82% vs 65%; difference = 17%; 95% CI, 7.8%-25.1%; χ21 = 12.2; P &lt; .001).The chatbot performed best on questions in the retina category (77% correct) and poorest in the neuro-ophthalmology category (58% correct). The chatbot demonstrated intermediate performance on questions from the ocular oncology (72% correct), pediatric ophthalmology (68% correct), uveitis (67% correct), and glaucoma (61% correct) categories.Conclusions and RelevanceIn this study, the recent version of the chatbot accurately responded to approximately two-thirds of multiple-choice questions pertaining to ophthalmic cases based on imaging interpretation. The multimodal chatbot performed better on questions that did not rely on the interpretation of imaging modalities. As the use of multimodal chatbots becomes increasingly widespread, it is imperative to stress their appropriate integration within medical contexts.

show abstract

Section: Discussioncontrasting

confidence: 57%

Section: Discussionmentioning

confidence: 99%

Accuracy of an Artificial Intelligence Chatbot’s Interpretation of Clinical Ophthalmic Images

Mihalache,

Huang,

Popovic

et al. 2024

JAMA Ophthalmol

View full text Add to dashboard Cite

show abstract

“…This could potentially reduce the cost of patient follow-up and expand accessibility to diagnostic support in underdeveloped areas that only have access to fundus photography. However, further validation and real-world application testing of the system are necessary to confirm its potential benefits [19][20][21].…”

Section: The Related Work Of Classification Fundus Photographsmentioning

confidence: 99%

Classification of Color Fundus Photographs Using Fusion Extracted Features and Customized CNN Models

Wang

et al. 2023

Healthcare

View full text Add to dashboard Cite

This study focuses on overcoming challenges in classifying eye diseases using color fundus photographs by leveraging deep learning techniques, aiming to enhance early detection and diagnosis accuracy. We utilized a dataset of 6392 color fundus photographs across eight disease categories, which was later augmented to 17,766 images. Five well-known convolutional neural networks (CNNs)—efficientnetb0, mobilenetv2, shufflenet, resnet50, and resnet101—and a custom-built CNN were integrated and trained on this dataset. Image sizes were standardized, and model performance was evaluated via accuracy, Kappa coefficient, and precision metrics. Shufflenet and efficientnetb0demonstrated strong performances, while our custom 17-layer CNN outperformed all with an accuracy of 0.930 and a Kappa coefficient of 0.920. Furthermore, we found that the fusion of image features with classical machine learning classifiers increased the performance, with Logistic Regression showcasing the best results. Our study highlights the potential of AI and deep learning models in accurately classifying eye diseases and demonstrates the efficacy of custom-built models and the fusion of deep learning and classical methods. Future work should focus on validating these methods across larger datasets and assessing their real-world applicability.

show abstract

Artificial Intelligence and Optometry: Transforming Practice and Patient Care

Majithia,

Thakur

2024

Current Practices in Ophthalmology

View full text Add to dashboard Cite

Application effect of an artificial intelligence-based fundus screening system: evaluation in a clinical setting and population screening

Cited by 7 publications

References 35 publications

Accuracy of an Artificial Intelligence Chatbot’s Interpretation of Clinical Ophthalmic Images

Accuracy of an Artificial Intelligence Chatbot’s Interpretation of Clinical Ophthalmic Images

Classification of Color Fundus Photographs Using Fusion Extracted Features and Customized CNN Models

Artificial Intelligence and Optometry: Transforming Practice and Patient Care

Contact Info

Product

Resources

About