Computer-Aided Ear Diagnosis System Based on CNN-LSTM Hybrid Learning Framework for Video Otoscopy Examination

Abstract

Ear disorders are among the most common diseases treated in primary care, with a high percentage of non-relevant referrals. The conventional diagnostic procedure is done by a visual examination of the ear canal and tympanic membrane. Consequently, the accuracy of the diagnosis is affected by observer-observer variation, depending on the technical skill and experiences of the physician as well as on the subjective bias of the observer. This situation impacts the proper implementation of treatments, increases health costs, and can lead to serious health complications. To eliminate subjectivity and enhance diagnostic accuracy, we present a diagnostic tool for nine ear conditions in a computer-aided diagnosis scheme. We propose a hybrid learning framework based on convolutional and recurrent neural networks for video otoscopy analysis. The proposed method first extracts the deep features of each relevant frame from the video. Then, a Long Short-term Memory network is introduced to learn spatial sequential data by analyzing deep features for a certain time interval. We carried out the study in collaboration with the Clinical Hospital of the University of Chile and included 875 subjects in a period of 12 months (continuous). The experiments were conducted on a new video otoscopy dataset and showed high performance in terms of accuracy (98.15%), precision (91.94%), sensitivity (91.67%), specificity (98.96%), and F1-score (91.51%). To the best of our knowledge, the proposed system is capable of predicting more diagnoses of ear conditions known to date with high performance. Our system is designed to assist in a real otoscopy examination by analyzing a sequence of images instead of a still image as previous state-of-the-art works. This advantage allows it to provide a comprehensive diagnosis of both eardrum and ear canal diseases.INDEX TERMS Computer-aided diagnosis, convolutional neural network, deep learning, ear diseases, LSTM, otolaryngology, transfer learning.

Automated multi-class classification for prediction of tympanic membrane changes with deep learning models

“…An extended dataset with diverse disease patterns can be used to validate the generality and robustness of our classification and improve the prediction performance of TM changes. In the same vein, when applied to otoendoscopic video sequences [ 15 ], it can help overcome the bias of still image-based prediction. Cerumen, which was not included in this study, may limit the information on TMs required for diagnosis.…”

Color Dependence Analysis in a CNN-Based Computer-Aided Diagnosis System for Middle and External Ear Diseases

“…On the other hand, the recording ends once the specula have been removed from the patient's ear producing unnecessary frames. Assuming that the central frame of the video shows the tympanic membrane and (or) the ear canal [8], we calculate the histogram of such a frame and compare it to each of the frames in the video using the Kullback-Leibler divergence score. If the score falls below a certain threshold, the frame is discarded.…”

“…Due to the advent of advanced processing techniques and the large volume of digital data available today, new technology to assist medical diagnosis has been generated as an increasing tendency to reduce diagnostic error in the last few years [6]. Some studies [7][8][9] have demonstrated the benefit of using artificial intelligence (AI) to improve and diagnose safety for ENT pathologies of various types by achieving a diagnostic performance of over 90%.…”

“…Our previous work [8] proposed a scheme for otological diagnosis using RGB (red, green, and blue) images from the external auditory canal. The model achieved high performance for nine ear pathologies (over 90%).…”

Smartphone-based artificial intelligence using a transfer learning algorithm for the detection and diagnosis of middle ear diseases: A retrospective deep learning study

“…To overcome this diagnostic dilemma and improve diagnostic accuracy, several methods, such as decision trees, support vector machines (SVMs), neural networks and Bayesian decision approaches, have been used to train different learning models and predict whether an image corresponds to a normal ear or an otitis media case. 8 , 9 Myburgh et al. utilized a video endoscope to acquire images and a cloud server for pre-processing and entered the images into a smartphone-based neural network program for diagnosis.…”