CST: A Multitask Learning Framework for Colorectal Cancer Region Mining Based on Transformer

Abstract

Colorectal cancer is a high death rate cancer until now; from the clinical view, the diagnosis of the tumour region is critical for the doctors. But with data accumulation, this task takes lots of time and labor with large variances between different doctors. With the development of computer vision, detection and segmentation of the colorectal cancer region from CT or MRI image series are a great challenge in the past decades, and there still have great demands on automatic diagnosis. In this paper, we proposed a novel transfer learning protocol, called CST, that is, a union framework for colorectal cancer region detection and segmentation task based on the transformer model, which effectively constructs the cancer region detection and its segmentation jointly. To make a higher detection accuracy, we incorporate an autoencoder-based image-level decision approach that leverages the image-level decision of a cancer slice. We also compared our framework with one-stage and two-stage object detection methods; the results show that our proposed method achieves better results on detection and segmentation tasks. And this proposed framework will give another pathway for colorectal cancer screen by way of artificial intelligence.

Preoperative CT-based deep learning model for predicting overall survival in patients with high-grade serous ovarian cancer

“…More recently, to overcome these limitations of CNN in computer vision problems, equipped with the Transformer architecture, Vision Transformer (ViT) was proposed to model long-range dependency among pixels through the self-attention mechanism (12), and has been demonstrated the state-of-the-art (SOTA) performance in a variety of vision tasks including object detection (30), classification (13), segmentation (31), and so on. At present, in the field of cancer, there are several researches using ViT for classify tasks (32-34) and cancer region detection and segmentation tasks (35). As far as we know, our study is the first to attempt to apply ViT for the survival prediction of HGSOC.…”

MRI‐Based Artificial Intelligence in Rectal Cancer

“…Although independent segmented networks have proven to be effective, there is also evidence that the multitasking model also plays an important role in the segmentation of colorectal tumors. Sui et al 64 proposed a novel transfer learning protocol called CST, that is, a union framework for colorectal cancer region detection and segmentation tasks based on the transformer model, which effectively achieved cancer region detection and segmentation jointly (Fig. 3c).…”

“…63 Figure 3c shows the different segmentation algorithms in MRI tumor segmentation tasks. 64 drawing requires trained professionals, making both time and cost consuming. The issue has accelerated unsupervised and self-supervised methods including deep learning, which do not require time-consuming labeling.…”

“…Yellow color is the overlap of green and red contours.Figure3bshows that three-dimensional (3D) surface renderings of the (red) bladder, (yellow) rectum, and (cyan) sigmoid segmentations by (left) the proposed method and (right) expert radiation oncologist's manual segmentation on MR1 63. Figure3cshows the different segmentation algorithms in MRI tumor segmentation tasks 64.…”