Applying Deep Learning with Convolutional Neural Networks to Laryngoscopic Imaging for Real-time Automated Segmentation and Classification of Vocal Cord Leukoplakia

Abstract

Objectives The study was to apply deep learning (DL) with convolutional neural networks (CNNs) to laryngoscopic imaging for assisting in real-time automated segmentation and classification of vocal cord leukoplakia. Methods This was a single-center retrospective diagnostic study included 216 patients who underwent laryngoscope and pathological examination from October 1, 2018 through October 1, 2019. Lesions were classified as nonsurgical group (NSG) and surgical group (SG) according to pathology. All selected images of vocal cord leukoplakia were annotated independently by 2 expert endoscopists and divided into a training set, a validation set, and a test set in a ratio of 6:2:2 for training the model. Results Among the 260 lesions identified in 216 patients, 2220 images from narrow band imaging (NBI) and 2144 images from white light imaging (WLI) were selected. For segmentation, the average intersection-over-union (IoU) value exceeded 70%. For classification, the model was able to classify the surgical group (SG) by laryngoscope with a sensitivity of 0.93 and specificity of 0.94 in WLI, and a sensitivity of 0.99 and specificity of 0.97 in NBI. Moreover, this model achieved a mean average precision (mAP) of 0.81 in WLI and 0.92 in NBI with an IoU> 0.5. Conclusions The study found that a model developed by applying DL with CNNs to laryngoscopic imaging results in high sensitivity, specificity, and mAP for automated segmentation and classification of vocal cord leukoplakia. This finding shows promise for the application of DL with CNNs in assisting in accurate diagnosis of vocal cord leukoplakia from WLI and NBI.

Convolutional neural network-based vocal cord tumor classification technique for home-based self-prescreening purpose

“…In addition, Cho et al reported the following two related papers: in one study, they applied four CNN models (six-layer CNN, VGG16, Inception-V3 and Xception) to laryngoscopic vocal fold images to classify the image into abnormal and normal [ 15 ], and in the other study, they applied four CNN models (VGG16, Inception-V3, MobileNet-V2 and EfficientNet-B0) to classify laryngeal diseases (cysts, nodules, polyps, leukoplakia, papillomas, Reinke’s edema, granulomas, palsies and normal) [ 16 ]; You et al applied 13 CNN models (AlexNet, four VGG models, three ResNet models, three DenseNet models, Inception-V3, and the proposed) to classify laryngeal leukoplakia (inflammatory keratosis, mild/moderate/severe dysplasia, and squamous cell carcinoma) using white-light endoscopy images [ 17 ]; Eggert et al applied DenseNet models to classify hyperspectral images of laryngeal, hypopharyngeal, and oropharyngeal mucosa into abnormal and normal [ 18 ]. Moreover, Hu et al applied Mask R-CNN with ResNet-50 backbone to two types of laryngoscopic imaging (narrow-band imaging and white-light imaging) for automated real-time segmentation and classification of vocal cord leukoplakia to classify the lesions into surgical and non-surgical groups [ 19 ]; Yan et al applied the Faster R-CNN model to laryngoscopic images of vocal lesions to screen for laryngeal carcinoma [ 20 ]; Kim et al applied the Mask R-CNN model to laryngoscopic images for real-time segmentation of laryngeal mass around the vocal cord [ 21 ]; Cen et al applied three CNN models (Faster R-CNN, Yolo V3, and SSD) to detect laryngeal tumors in endoscopic images (vocal fold, tumor, surgical tools, and other laryngeal tissues) [ 22 ]; Azam et al applied up to nine Yolo models to laryngoscopic video for real-time detection of laryngeal squamous cell carcinoma in both white-light and narrow-band imaging [ 23 ]. Among these previous studies on vocal area disease detection, eight [ 11 – 18 ] used AI models for classification and, therefore, were not able to provide information about the tumor-suspicious positions in the image.…”

“…Among these previous studies on vocal area disease detection, eight [ 11 – 18 ] used AI models for classification and, therefore, were not able to provide information about the tumor-suspicious positions in the image. Similar to the current study, five other studies [ 19 – 23 ] used AI models for object detection that can provide tumor-suspicious positions around the vocal cords; however, they commonly used only single-group disease images, such as vocal cord leukoplakia [ 19 ], laryngeal carcinoma [ 20 , 23 ], laryngeal mass [ 21 ], and cancer [ 22 ].…”