Automatic classification of informative laryngoscopic images using deep learning

Abstract

Objective: This study aims to develop and validate a convolutional neural network (CNN)-based algorithm for automatic selection of informative frames in flexible laryngoscopic videos. The classifier has the potential to aid in the development of computer-aided diagnosis systems and reduce data processing time for cliniciancomputer scientist teams. Methods: A dataset of 22,132 laryngoscopic frames was extracted from 137 flexible laryngostroboscopic videos from 115 patients. 55 videos were from healthy patients with no laryngeal pathology and 82 videos were from patients with vocal fold polyps. The extracted frames were manually labeled as informative or uninformative by two independent reviewers based on vocal fold visibility, lighting, focus, and camera distance, resulting in 18,114 informative frames and 4018 uninformative frames. The dataset was split into training and test sets. A pre-trained ResNet-18 model was trained using transfer learning to classify frames as informative or uninformative.Hyperparameters were set using cross-validation. The primary outcome was precision for the informative class and secondary outcomes were precision, recall, and F1-score for all classes. The processing rate for frames between the model and a human annotator were compared. Results:The automated classifier achieved an informative frame precision, recall, and F1-score of 94.4%, 90.2%, and 92.3%, respectively, when evaluated on a hold-out test set of 4438 frames. The model processed frames 16 times faster than a human annotator. Conclusion:The CNN-based classifier demonstrates high precision for classifying informative frames in flexible laryngostroboscopic videos. This model has the

Artificial Intelligence for Upper Aerodigestive Tract Endoscopy and Laryngoscopy: A Guide for Physicians and State‐of‐the‐Art Review

“…28 In more recent work, Yao et al described the development of a DL model for a simple classification between informative versus noninformative laryngoscopic frames, which was assessed on a larger dataset of 22,132 images. 27 The model achieved an informative frame Precision, Recall, and F1 score of 94.4%, 90.2%, and 92.3%, respectively, demonstrating to be potentially useful in clinical applications. In this regard, the advantages offered by the automatic assessment of frame quality can lead to the development of novel tools able to help physicians to perform more thorough and informative examinations also possibly providing an objective quality assessment during the procedure.…”

“…One of the most frequent attempts to explore the interpretability of these DL models is to implement a gradient-weighted class activation map which can visualize the contribution of each pixel to the algorithm prediction. 5,6,[8][9][10][17][18][19]27 This graphical representation can give an insight into how these models work, but we are still far from fully understanding their learning mechanisms.…”

Artificial Intelligence and Laryngeal Cancer: From Screening to Prognosis: A State of the Art Review

“…As new technologies arise, automatic classification of informative vs noninformative frames based on ML technology will be needed to develop models that can be integrated into clinical practice. This will be key to avoid the resource‐intensive process of data preparation and manual keyframe extraction in laryngoscopy videos 20,21 …”

“…This will be key to avoid the resource-intensive process of data preparation and manual keyframe extraction in laryngoscopy videos. 20,21 Radiomics. Radiomics refers to the extraction of quantitative features from digital radiology images and subsequent data analysis for hypothesis generation and testing.…”

A Novel Framework of Manifold Learning Cascade-Clustering for the Informative Frame Selection

“…The dataset contains 22,000 laryngoscopic frames. It is the second dataset announced for the problem of informative frame selection and the largest one [ 40 ].…”