Fully convolutional networks in multimodal nonlinear microscopy images for automated detection of head and neck carcinoma: Pilot study

Abstract

Background: A fully convolutional neural networks (FCN)-based automated image analysis algorithm to discriminate between head and neck cancer and noncancerous epithelium based on nonlinear microscopic images was developed. Methods: Head and neck cancer sections were used for standard histopathology and co-registered with multimodal images from the same sections using the combination of coherent anti-Stokes Raman scattering, two-photon excited fluorescence, and second harmonic generation microscopy. The images analyzed with semantic segmentation using a FCN for four classes: cancer, normal epithelium, background, and other tissue types. Results: A total of 114 images of 12 patients were analyzed. Using a patch score aggregation, the average recognition rate and an overall recognition rate or the four classes were 88.9% and 86.7%, respectively. A total of 113 seconds were needed to process a whole-slice image in the dataset. Conclusion: Multimodal nonlinear microscopy in combination with automated image analysis using FCN seems to be a promising technique for objective differentiation between head and neck cancer and noncancerous epithelium.coherent anti-stokes Raman scattering, convolutional neural networks, diagnostics, digital pathology, head and neck cancer, image analysis, second-harmonic generation, semantic segmentation, spectral histopathology, two-photon excited fluorescence

Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology

“…This attribute enables a FCN to make pixel-level predictions with a possible advantage over a CNN, which learns from repetitive features that occur throughout the entire image. In digital pathology, FCNs have been used by Rodner et al 98 to differentiate cancerous regions from non-malignant epithelium in histopathology images of head and neck cancer specimens. Using co-registered H&E images with multimodal microscopy techniques, the FCN was used to segment the WSIs into four classes: cancer, non-malignant epithelium, background and other tissue.…”

Multimodal Imaging Based on Vibrational Spectroscopies and Mass Spectrometry Imaging Applied to Biological Tissue: A Multiscale and Multiomics Review

“…This is a valuable research investment as it can pinpoint limitations and test many of the assumptions in multimodal imaging, thereby improving sample preparations. Artificial intelligence (AI), through ML and deep learning, is taking on an increasingly important role in MSI 193 and vibrational spectroscopies, [307][308][309] particularly as it pertains to data analysis. Also there is increasing interests in image analysis through AI which could be impactful in pathology.…”

Rapid histology of laryngeal squamous cell carcinoma with deep-learning based stimulated Raman scattering microscopy

“…Clinical SRS histology has recently stepped forward to the operating room with a fiber-laser based portable system, generating reliable intraoperative histological results on unprocessed surgical brain tissues 26,27. Despite previous efforts on coherent Raman scattering histopathology on a few types of tissues and diseases 8-13,26,28-33, the potential for SRS to diagnose larynx tissues has never been rigorous investigated.…”