Integrative Data Augmentation with U-Net Segmentation Masks Improves Detection of Lymph Node Metastases in Breast Cancer Patients

Abstract

Deep learning models have potential to improve performance of automated computer-assisted diagnosis tools in digital histopathology and reduce subjectivity. The main objective of this study was to further improve diagnostic potential of convolutional neural networks (CNNs) in detection of lymph node metastasis in breast cancer patients by integrative augmentation of input images with multiple segmentation channels. For this retrospective study, we used the PatchCamelyon dataset, consisting of 327,680 histopathology images of lymph node sections from breast cancer. Images had labels for the presence or absence of metastatic tissue. In addition, we used four separate histopathology datasets with annotations for nucleus, mitosis, tubule, and epithelium to train four instances of U-net. Then our baseline model was trained with and without additional segmentation channels and their performances were compared. Integrated gradient was used to visualize model attribution. The model trained with concatenation/integration of original input plus four additional segmentation channels, which we refer to as ConcatNet, was superior (AUC 0.924) compared to baseline with or without augmentations (AUC 0.854; 0.884). Baseline model trained with one additional segmentation channel showed intermediate performance (AUC 0.870-0.895). ConcatNet had sensitivity of 82.0% and specificity of 87.8%, which was an improvement in performance over the baseline (sensitivity of 74.6%; specificity of 80.4%). Integrated gradients showed that models trained with additional segmentation channels had improved focus on particular areas of the image containing aberrant cells. Augmenting images with additional segmentation channels improved baseline model performance as well as its ability to focus on discrete areas of the image.

An Intelligent Diagnostic Model for Melasma Based on Deep Learning and Multimode Image Input

“…In a study by Tschandl et al, both dermoscopic images and clinical close-ups were used to train the network; the combination was found to acquire a better result than the individual modalities [34]. Jin et al found that multiple extracted histological features, including nuclei, mitosis, epithelial, and tubular cells, could further improve the detection of lymph node metastasis in patients with breast cancer [35]. This study used a multichannel image input method to fuse multiple mode images in the VISIA system and discovered that it could improve the accuracy of our network to some extent.…”

Quantitative X-ray phase contrast computed tomography with grating interferometry

“…A potential application in this direction could be prehistology GBPC-CT, where quantitative GBPC-CT data could be used for advanced tissue characterization leading to more precise and efficient sectioning in histology. Having the ability to access absolute quantitative GBPC-CT data could be used for deep learning and could also lead to improved histological diagnosis [184][185][186].…”

The adoption of deep learning interpretability techniques on diabetic retinopathy analysis: a review

“…The authors adopted the CAM approach to identify patterns that influenced the model’s judgment on whether bacterial or viral pneumonia was present in the lungs. [ 34 ] used the IG technique to illustrate model attribution in the biomedical field of a breast cancer diagnosis. They claimed that models with more segmentation channels were better at focusing on specific parts of the image containing abnormal cells.…”