TuNet: End-to-End Hierarchical Brain Tumor Segmentation Using Cascaded Networks

Self Cite

Purpose When using convolutional neural networks (CNNs) for segmentation of organs and lesions in medical images, the conventional approach is to work with inputs and outputs either as single slice [two‐dimensional (2D)] or whole volumes [three‐dimensional (3D)]. One common alternative, in this study denoted as pseudo‐3D, is to use a stack of adjacent slices as input and produce a prediction for at least the central slice. This approach gives the network the possibility to capture 3D spatial information, with only a minor additional computational cost. Methods In this study, we systematically evaluate the segmentation performance and computational costs of this pseudo‐3D approach as a function of the number of input slices, and compare the results to conventional end‐to‐end 2D and 3D CNNs, and to triplanar orthogonal 2D CNNs. The standard pseudo‐3D method regards the neighboring slices as multiple input image channels. We additionally design and evaluate a novel, simple approach where the input stack is a volumetric input that is repeatably convolved in 3D to obtain a 2D feature map. This 2D map is in turn fed into a standard 2D network. We conducted experiments using two different CNN backbone architectures and on eight diverse data sets covering different anatomical regions, imaging modalities, and segmentation tasks. Results We found that while both pseudo‐3D methods can process a large number of slices at once and still be computationally much more efficient than fully 3D CNNs, a significant improvement over a regular 2D CNN was only observed with two of the eight data sets. triplanar networks had the poorest performance of all the evaluated models. An analysis of the structural properties of the segmentation masks revealed no relations to the segmentation performance with respect to the number of input slices. A post hoc rank sum test which combined all metrics and data sets yielded that only our newly proposed pseudo‐3D method with an input size of 13 slices outperformed almost all methods. Conclusion In the general case, multislice inputs appear not to improve segmentation results over using 2D or 3D CNNs. For the particular case of 13 input slices, the proposed novel pseudo‐3D method does appear to have a slight advantage across all data sets compared to all other methods evaluated in this work.

Section: B Contributionsmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

Section: B Quantitative Analysismentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of multislice inputs to convolutional neural networks for medical image segmentation

Grimbergen

Nyholm

et al. 2020

Self Cite

“…The generative adversarial network (GAN) 28 synthesizes high‐contrast images to transform the intensity distribution of brain lesions in its internal subregions. The multistep cascade (MSC) 29 and TuNet 30 considers the hierarchical topology of the brain tumor substructures and segments the substructures from coarse to fine. The VAE 31 adds a variational autoencoder branch in the encoder‐decoder structure to regularize the shared decoder and impose additional constraints on its layers.…”

Section: Resultsmentioning

confidence: 99%

Correcting and reweighting false label masks in brain tumor segmentation

Cheng

Ji²,

He³

2020

Purpose Recently, brain tumor segmentation has made important progress. However, the quality of manual labels plays an important role in the performance, while in practice, it could vary greatly and in turn could substantially mislead the learning process and decrease the accuracy. We need to design a mechanism to combine label correction and sample reweighting to improve the effectiveness of brain tumor segmentation. Methods We propose a novel sample reweighting and label refinement method, and a novel three‐dimensional (3D) generative adversarial network (GAN) is introduced to combine these two models into an united framework. Results Extensive experiments on the BraTS19 dataset have demonstrated that our approach obtains competitive results when compared with other state‐of‐the‐art approaches when handling the false labels in brain tumor segmentation. Conclusions The 3D GAN‐based approach is an effective approach to handle false label masks by simultaneously applying label correction and sample reweighting. Our method is robust to variations in tumor shape and background clutter.

“…Some approaches 7–10 segment the tumor in a coarse‐to‐fine manner, using first 3D fully convolutional networks (FCNs) to roughly detect a candidate region and then other FCNs to perform pixel‐wise segmentation based on the candidate region, but the accuracy of segmentation depends on the detection process. A CA‐CNN, 11 CU‐Net, 1 multistep cascaded network (MCN), 10 and TuNet 12 segment different tumors in sequence using other kinds of tumor regions as a prior for the next kind of tumor segmentation. However, all the subnetworks are independent and trained one‐by‐one.…”

Section: Introductionmentioning

confidence: 99%

Spatial‐channel relation learning for brain tumor segmentation

Cheng

Ji²,

Ding

2020

Purpose: Recently, research on brain tumor segmentation has made great progress. However, ambiguous patterns in magnetic resonance imaging data and linear fusion omitting semantic gaps between features in different branches remain challenging. We need to design a mechanism to fully utilize the similarity within the spatial space and channel space and the correlation between these two spaces to improve the result of volumetric segmentation. Methods: We propose a revised cascade structure network. In each subnetwork, a context exploitation module is introduced between the encoder and decoder, in which the dual attention mechanism is adopted to learn the information within the spatial space and channel space, and space interaction learning is employed to model the relation between the spatial and channel spaces. Results: Extensive experiments on the BraTS19 dataset have evaluated that our approach improves the dice coefficient (DC) by a margin of 2.1, 2.0, and 1.4 for whole tumor (WT), tumor core (TC), and enhancing tumor (ET), respectively, obtaining results competitive with the state-of-art approaches working on brain tumor segmentation. Conclusions: Context exploitation in the embedding feature spaces, including intraspace relations and interspace relations, can effectively model dependency in semantic features and alleviate the semantic gap in multimodel data. Our approach is also robust to variations in different modality.