Signal to noise ratio quantifies the contribution of spectral channels to classification of human head and neck tissues ex vivo using deep learning and multispectral imaging

Abstract

. Significance Accurate identification of tissues is critical for performing safe surgery. Combining multispectral imaging (MSI) with deep learning is a promising approach to increasing tissue discrimination and classification. Evaluating the contributions of spectral channels to tissue discrimination is important for improving MSI systems. Aim Develop a metric to quantify the contributions of individual spectral channels to tissue classification in MSI. Approach MSI was integrated into a digital operating microscope with three sensors and seven illuminants. Two convolutional neural network (CNN) models were trained to classify 11 head and neck tissue types using white light (RGB) or MSI images. The signal to noise ratio (SNR) of spectral channels was compared with the impact of channels on tissue classification performance as determined using CNN visualization methods. Results Overall tissue classification accuracy was higher with use of MSI images compared with RGB images, both for classification of all 11 tissue types and binary classification of nerve and parotid ( ). Removing spectral channels with reduced tissue classification accuracy. Conclusions The spectral channel SNR is a useful metric for both understanding CNN tissue classification and quantifying the contributions of different spectral channels in an MSI system.

End‐to‐end deep learning classification of vocal pathology using stacked vowels

“…A longstanding challenge in the practical application of deep learning models is interpretability. Visualization techniques, such as saliency maps and occlusion maps, 35 are well established in computer vision for providing insight into the specific regions of images that are used for image classification and detection 36–38 . Approaches using spectrograms 19 could allow application of visualization techniques to delineate areas of spectrograms that are influential on vocal pathology predictions.…”

“…Visualization techniques, such as saliency maps and occlusion maps, 35 are well established in computer vision for providing insight into the specific regions of images that are used for image classification and detection. 36 , 37 , 38 Approaches using spectrograms 19 could allow application of visualization techniques to delineate areas of spectrograms that are influential on vocal pathology predictions. Visualization methods are also being developed for audio deep neural networks and may be more available in the future.…”

Switchable LED-based laparoscopic multispectral system for rapid high-resolution perfusion imaging

“…For the latter, it was recommended to use multiple LEDs of each wavelength 19 . Systems have been presented with identical LEDs opposite to each other 20 or arranged 90 deg apart from each other 20 , 21 . The results will be lower light losses and better mixing of individual wavelengths, the ability to visualize smaller gradients of tissue hemoglobin content or oxygenation, and the use of the system at larger measuring distances.…”

“… 19 Systems have been presented with identical LEDs opposite to each other 20 or arranged 90 deg apart from each other. 20 , 21 The results will be lower light losses and better mixing of individual wavelengths, the ability to visualize smaller gradients of tissue hemoglobin content or oxygenation, and the use of the system at larger measuring distances. However, multiple identical LEDs will be associated with a reduction of the available spectral range, higher costs, or bulky setups.…”

“…A cut-off value of 20 dB was determined by Liu et al. 21 in investigating the contribution of spectral channels to the accuracy of tissue classification algorithms depending on their SNR. Similarly, Mühle et al.…”