Sketch-Supervised Histopathology Tumour Segmentation: Dual CNN-Transformer With Global Normalised CAM

Abstract

Deep learning methods are frequently used in segmenting histopathology images with high-quality annotations nowadays. Compared with well-annotated data, coarse, scribbling-like labelling is more cost-effective and easier to obtain in clinical practice. The coarse annotations provide limited supervision, so employing them directly for segmentation network training remains challenging. We present a sketch-supervised method, called DCTGN-CAM, based on a dual CNN-Transformer network and a modified global normalised class activation map. By modelling global and local tumour features simultaneously, the dual CNN-Transformer network produces accurate patchbased tumour classification probabilities by training only on lightly annotated data. With the global normalised class activation map, more descriptive gradient-based representations of the histopathology images can be obtained, and inference of tumour segmentation can be performed with high accuracy. Additionally, we collect a private skin cancer dataset named BSS, which contains fine and coarse annotations for three types of cancer. To facilitate reproducible performance comparison, experts are also invited to label coarse annotations on the public liver cancer dataset PAIP2019. On the BSS dataset, our DCTGN-CAM segmentation outperforms the state-of-the-art methods and achieves 76.68 % IOU and 86.69 % Dice scores on the sketch-based tumour segmentation task. On the PAIP2019 dataset, our method achieves a Dice gain of 8.37 % compared with U-Net as the baseline network. The dataset, annotation and code will be published at https://github.com/skdarkless/DCTGN-CAM.

Multi-Convolutional Neural Network-Based Diagnostic Software for the Presumptive Determination of Non-Dermatophyte Molds

“…One way of CNN decision-making visualization is by using the Grad-CAM method. Class activation maps can be used to interpret the prediction decision made by the convolutional neural network, and they can be implemented in various systems, like object detection [37], histopathology segmentation [38], and rotating machinery fault diagnosis [39]. The Grad-CAM technique highlights parts of the image that represent the recognized pattern by extracting gradients of the target notion (molds in our model) [40].…”

Guest Editorial Computational Mathematics Modeling in Cancer Analysis

“…The sixth paper by Yilong Li et al [9] focus on Weakly supervised segmentation model. it presents a sketchsupervised method, called DCTGNCAM, based on a dual CNN-Transformer network and a modified global normalized class activation map.…”

A target region extraction method for ultrasound medical images based on improvedPRIDNetandUCTransNet

“…In recent years, the success of Transformer architecture 26 in natural images has spurred research in the domain of medical imaging as well 27,28 . The prevailing network design strategy involves embedding Transformers into U‐Net frameworks, exemplified by architectures like TransUnet, 29 Swin‐Unet, 30 H2Former, 31 UCTransNet, 32 and others 33,34 . Nevertheless, when addressing the challenge of inconspicuous target regions arising from noise and artifacts in ultrasound images, the aforementioned methods are primarily addressed through preprocessing techniques like normalization and the establishment of pixel enhancement thresholds.…”

“…27,28 The prevailing network design strategy involves embedding Transformers into U-Net frameworks, exemplified by architectures like TransUnet, 29 Swin-Unet, 30 H2Former, 31 UCTransNet, 32 and others. 33,34 Nevertheless, when addressing the challenge of inconspicuous target regions arising from noise and artifacts in ultrasound images, the aforementioned methods are primarily addressed through preprocessing techniques like normalization and the establishment of pixel enhancement thresholds. However, such processing often leads to the loss of image edges and internal information, presenting challenges to the accurate segmentation of the target region.…”