EFNet: evidence fusion network for tumor segmentation from PET-CT volumes

Abstract

Precise delineation of target tumor from positron emission tomography-computed tomography (PET-CT) is a key step in clinical practice and radiation therapy. PET-CT co-segmentation actually uses the complementary information of two modalities to reduce the uncertainty of single-modal segmentation, so as to obtain more accurate segmentation results. At present, the PET-CT segmentation methods based on fully convolutional neural network (FCN) mainly adopt image fusion and feature fusion. The current fusion strategies do not consider the uncertainty of multi-modal segmentation and complex feature fusion consumes more computing resources, especially when dealing with 3D volumes. In this work, we analyze the PET-CT co-segmentation from the perspective of uncertainty, and propose evidence fusion network (EFNet). The network respectively outputs PET result and CT result containing uncertainty by proposed evidence loss, which are used as PET evidence and CT evidence. Then we use evidence fusion to reduce uncertainty of single-modal evidence. The final segmentation result is obtained based on evidence fusion of PET evidence and CT evidence. EFNet uses the basic 3D U-Net as backbone and only uses simple unidirectional feature fusion. In addition, EFNet can separately train and predict PET evidence and CT evidence, without the need for parallel training of two branch networks. We do experiments on the soft-tissue-sarcomas and lymphoma datasets. Compared with 3D U-Net, our proposed method improves the Dice by 8% and 5% respectively. Compared with the complex feature fusion method, our proposed method improves the Dice by 7% and 2% respectively. Our results show that in PET-CT segmentation methods based on FCN, by outputting uncertainty evidence and evidence fusion, the network can be simplified and the segmentation results can be improved.

Multi-modal tumor segmentation methods based on deep learning: a narrative review

“…In the HECKTOR dataset, where the axial size of PET and CT images is different, some methods resample PET/CT images to 1×1×1 mm in each direction and crop them to 144×144×144 size ( 23 , 24 , 35 ). For the STS dataset, where the PET axial size is 128×128, and the CT axial size is 256×256, some works ( 36 , 37 ) resize CT to match PET size to 128×128, or linearly interpolate PET to CT size ( 38 ).…”

DMCT-Net: dual modules convolution transformer network for head and neck tumor segmentation in PET/CT

“…Given the similarities between OPC and NPC, we also compared the segmentation methods for NPC. In addition, we compared with other tumor segmentation methods based on FDG-PET/CT datasets (Diao et al 2021, Fu et al 2021 as well as some general medical image segmentation methods (Isensee et al 2021, Valanarasu and Patel 2022, As shown in table 1, excluding nnUNet (Isensee et al 2021), the proposed method was superior to all other methods in terms of DSC, precision, and sensitivity, indicating that our method had good segmentation performance and could effectively improve the accuracy of tumor segmentation. Furthermore, the proposed model had poor performance in the HD95 segmentation metric, indicating that the proposed method was not sensitive to tumor boundary segmentation.…”

“…The models performed well on photo-based datasets, which have certain clinical application values in specific fields. Compared with the methods proposed by Fu et al (2021), Isensee et al (2021), andDiao et al (2021), our method had relatively low requirements for the experimental environment and equipment. Compared with other comparison methods, the proposed method had relatively high segmentation performance.…”

“…For example, Fu et al (2021) designed a deep learning network architecture with a multimodal spatial attention module to segment lung tumors and soft tissue tumors in FDG-PET/CT images. Diao et al (2021) analyzed the FDG-PET/CT co-segmentation problem from the uncertainty perspective and designed an evidence fusion network. They conducted experiments on soft tissue tumor and lymphoma datasets.…”

“…First, existing methods usually only considered one or two of multi-scale information, local information, and global information, and lack the effective fusion and utilization of all the above information, such as U-Net variant network (Huang et al 2018, Groendahl et al 2021, dense connection convolutional network variants (Guo et al 2019. Second, in the down-sampling process of the encoding and decoding network, the network only focused on the salient features or contextual features and lacked multi-angle extraction and adaptive fusion of features to reduce the feature dilution (Huang et al 2018, Diao et al 2021, Fu et al 2021. Finally, for the multimodal images of FDG-PET/CT, the 3D spatial feature provided the tumor position information, and the channel feature integrated the complementary advantages of multiple modalities.…”