Triaging ophthalmology outpatient referrals with machine learning: A pilot study

Abstract

Importance Triaging of outpatient referrals to ophthalmology services is required for the maintenance of patient care and appropriate resource allocation. Machine learning (ML), in particular natural language processing, may be able to assist with the triaging process. Background To determine whether ML can accurately predict triage category based on ophthalmology outpatient referrals. Design Retrospective cohort study. Participants The data of 208 participants was included in the project. Methods The synopses of consecutive ophthalmology outpatient referrals at a tertiary hospital were extracted along with their triage categorizations. Following pre‐processing, ML models were applied to determine how accurately they could predict the likely triage categorization allocated. Data was split into training and testing sets (75%/25% split). ML models were tested on an unseen test set, after development on the training dataset. Main Outcome Measure Area under the receiver operator curve (AUC) for category one vs non‐category one classification. Results For the main outcome measure, convolutional neural network (CNN) provided the best AUC (0.83) and accuracy on the test set (0.81), with the artificial neural network (AUC 0.81 and accuracy 0.77) being the next best performing model. When the CNN was applied to the classification task of identifying which referrals should be allocated a category one vs category two vs category three priority, a lower accuracy was achieved (0.65). Conclusions and Relevance ML may be able to accurately assist with the triaging of ophthalmology referrals. Future studies with data from multiple centres and larger sample sizes may be beneficial.

Identification of the high-risk area for schistosomiasis transmission in China based on information value and machine learning: a newly data-driven modeling attempt

“…The sample data were randomly divided into two parts: 75% as training samples for model construction, and 25% as test samples to evaluate the accuracy, referred to relevant literature [ 19 ]. A confusion matrix was used to reflect the comprehensive performance of the models (Table 2 ).…”

A Proposed Framework for Machine Learning-Aided Triage in Public Specialty Ophthalmology Clinics in Hong Kong

“…An end-to-end clinical framework for deploying intelligent systems should include data collection and pre-processing protocols, data annotation, system architecture, deployment strategies and evaluation metrics. Our proposed framework uses colour fundus photos (CFPs) and referral synopses as the inputs since pilot studies have shown their utility in triaging [ 41 ] and grading important diseases, including diabetic retinopathy (DR) [ 14 ], age-related macular degeneration (AMD) [ 5 ] and urgent cases such as glaucoma [ 6 ]. Research in emergency department triaging has also found that textual data is significant for the accuracy of a triaging ML network [ 31 ].…”

“…Textual data processing includes tokenisation and padding. Previous research [ 41 ] demonstrated good accuracy with sub-word tokenisation and word-sequence-independent methods (e.g. ANN, random forest), achieving an AUC of 0.83 and accuracy of 0.81 in categorising urgent vs non-urgent ophthalmology referrals.…”

“…The current SOTA on image classification is EfficientNet [ 42 ], whose pre-trained weights on ImageNet [ 9 ] are publicly available. For NLU tasks, networks in the literature on biomedical texts span from convolutional neural networks [ 41 ] to transformers (e.g. BERT and its variants) [ 51 ].…”

Triaging Medical Referrals Based on Clinical Prioritisation Criteria Using Machine Learning Techniques

“…They have applications in many domains, including healthcare and medical research [ 2 , 3 ]. Machine learning and deep learning-driven approaches have been utilised for automation of the medical referral triaging process [ 4 , 5 ] or derivation of insights where unique or anomalous referrals can be detected among the triaged groups to support clinicians in appreciating the landscape of the past or present referrals [ 6 ].…”