3D U$$^2$$-Net: A 3D Universal U-Net for Multi-domain Medical Image Segmentation

Abstract

Fully convolutional neural networks like U-Net have been the state-of-the-art methods in medical image segmentation. Practically, a network is highly specialized and trained separately for each segmentation task. Instead of a collection of multiple models, it is highly desirable to learn a universal data representation for different tasks, ideally a single model with the addition of a minimal number of parameters steered to each task. Inspired by the recent success of multi-domain learning in image classification, for the first time we explore a promising universal architecture that handles multiple medical segmentation tasks and is extendable for new tasks, regardless of different organs and imaging modalities. Our 3D Universal U-Net (3D U 2 -Net) is built upon separable convolution, assuming that images from different domains have domain-specific spatial correlations which can be probed with channelwise convolution while also share cross-channel correlations which can be modeled with pointwise convolution. We evaluate the 3D U 2 -Net on five organ segmentation datasets. Experimental results show that this universal network is capable of competing with traditional models in terms of segmentation accuracy, while requiring only about 1% of the parameters. Additionally, we observe that the architecture can be easily and effectively adapted to a new domain without sacrificing performance in the domains used to learn the shared parameterization of the universal network. We put the code of 3D U 2 -Net into public domain. 4

A Noise-Robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions From CT Images

“…3. Inspired by the state-of-the-art performance of variants of U-Net [16], [36]- [38] with an encoder-decoder structure, we use a similar backbone but extend it with several important modules. First, differently from [16], [36]- [38] that only use max-pooling for down-sampling, we introduce a dual pooling, i.e., a concatenation of max-pooling and averagepooling as down-sampling, which has a lower information loss than a simple max-pooling.…”

“…Inspired by the state-of-the-art performance of variants of U-Net [16], [36]- [38] with an encoder-decoder structure, we use a similar backbone but extend it with several important modules. First, differently from [16], [36]- [38] that only use max-pooling for down-sampling, we introduce a dual pooling, i.e., a concatenation of max-pooling and averagepooling as down-sampling, which has a lower information loss than a simple max-pooling. Second, we replace the typical skip connection between the encoder and the decoder with a bridge layer (i.e., 1 × 1 convolution) to map the lowlevel features from the encoder to a lower dimension (i.e., the channel number is reduced by half) before concatenating them with high-level features from the decoder, in order to alleviate the semantic gap between the low-level and high-level features [39].…”

Label-Free Segmentation of COVID-19 Lesions in Lung CT

“…For a better performance on COVID-19 segmentation, we plan to extend our method to address the above limitations mainly in the following three aspects: 1) Modeling more 'non-COVID-19' contexts including other diseases; and 2) Exploring a better way of modeling low-intensity normal voxels as much as possible by mitigating the impact of noise with an array of denoising methods. 3) Creating a more effective synthetic 'lesions' generator for network learning by exploring different generation schemes, such as using a deeper hierarchy and a universal generation [71] by investigating cross-anatomy or even cross-modality possibilities. 4) Exploring the idea of metric learning such as Deep SVDD [41] to get tighter decision boundary.…”

Numerical Evaluation on Parametric Choices Influencing Segmentation Results in Radiology Images—A Multi-Dataset Study

“…We have seen in the literature [11][12][13][14], the most promising successor of FCN is 3D-UNet. Used widely for multi-class segmentation [52,53] and often referred to as a universal segmentation model, we decided to follow the same idea of employing 3D-UNet for multi-dataset segmentation. We used different combinations of the parameters including weight initialization, loss functions and optimizers keeping ReLU as the activation function considering two main reasons.According to the literature, ReLU performed well with the U-Net architecture [54,55] and is considered to be six times faster than sigmoid/tanh activation functions.…”