UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-Wise Perspective with Transformer

Abstract

Most recent semantic segmentation methods adopt a U-Net framework with an encoder-decoder architecture. It is still challenging for U-Net with a simple skip connection scheme to model the global multi-scale context: 1) Not each skip connection setting is effective due to the issue of incompatible feature sets of encoder and decoder stage, even some skip connection negatively influence the segmentation performance; 2) The original U-Net is worse than the one without any skip connection on some datasets. Based on our findings, we propose a new segmentation framework, named UCTransNet (with a proposed CTrans module in U-Net), from the channel perspective with attention mechanism. Specifically, the CTrans (Channel Transformer) module is an alternate of the U-Net skip connections, which consists of a sub-module to conduct the multi-scale Channel Cross fusion with Transformer (named CCT) and a sub-module Channel-wise Cross-Attention (named CCA) to guide the fused multi-scale channel-wise information to effectively connect to the decoder features for eliminating the ambiguity. Hence, the proposed connection consisting of the CCT and CCA is able to replace the original skip connection to solve the semantic gaps for an accurate automatic medical image segmentation. The experimental results suggest that our UCTransNet produces more precise segmentation performance and achieves consistent improvements over the state-of-the-art for semantic segmentation across different datasets and conventional architectures involving transformer or U-shaped framework. Code: https://github.com/McGregorWwww/UCTransNet.

FFUNet: A novel feature fusion makes strong decoder for medical image segmentation

“…Table 3 demonstrates that our FFUNet achieves the best performance with DSC 80.63% and HD 18.15 mm. Compared with TransUNet [15], UCTransNet [17] and SwinUNet [14], although the proposed method only got slightly higher than prior arts on the DSC evaluation metric, we gained 13.5 mm in term of HD, which indicates that FFUNet can improve the edge predictions in medical image segmentation tasks.…”

“…To verify the overall segmentation performance of the proposed FFUNet, we conduct main experiments on the Synapse multi‐organ segmentation dataset compared with state of the arts, specifically methods of TransUNet [15], UCTransNet [17] and SwinUNet [14]. The results are presented in Table 3 where the best results are bold‐faced.…”

“…Following UCTransNet [17], we employ the dice coefficient (Dice) and Intersection over Union (IoU) as evaluation metrics to evaluate the performance of FFUNet on GlaS and MoNuSeg datasets. FFUNet‐nopre denotes training FFUNet without using any pre‐trained weights.…”

Online Easy Example Mining for Weakly-supervised Gland Segmentation from Histology Images

“…Moreover, we design a powerful framework for weakly-supervised gland segmentation and report its fully-supervised result of the segmentation stage. With this strong baseline, our method excels many previous fully supervised methods [13,14,15] on the GlaS [11] dataset, notably outperforming the prior best method [15] by 4.6% on mIoU. Importantly, even with such a high backbone, our proposed OEEM further increases the performance by around 2.0% mIoU in weakly settings.…”

“…It contains 165 images derived from 16 Hematoxylin, and Eosin (H&E) stained histological Whole Slide Images (WSIs) of stage T3 or T42 colorectal adenocarcinoma. Following previous works [13,14,15], we split the data into 85 training images and 80 test images, which are separated by patients as original dataset without patch shuffle. There is no classifiable image-level label, since glands exist in each image.…”