Simultaneous cosegmentation of tumors in PET‐CT images using deep fully convolutional networks

Abstract

Purpose To investigate the use and efficiency of 3‐D deep learning, fully convolutional networks (DFCN) for simultaneous tumor cosegmentation on dual‐modality nonsmall cell lung cancer (NSCLC) and positron emission tomography (PET)‐computed tomography (CT) images. Methods We used DFCN cosegmentation for NSCLC tumors in PET‐CT images, considering both the CT and PET information. The proposed DFCN‐based cosegmentation method consists of two coupled three‐dimensional (3D)‐UNets with an encoder‐decoder architecture, which can communicate with the other in order to share complementary information between PET and CT. The weighted average sensitivity and positive predictive values denoted as Scores, dice similarity coefficients (DSCs), and the average symmetric surface distances were used to assess the performance of the proposed approach on 60 pairs of PET/CTs. A Simultaneous Truth and Performance Level Estimation Algorithm (STAPLE) of 3 expert physicians’ delineations were used as a reference. The proposed DFCN framework was compared to 3 graph‐based cosegmentation methods. Results Strong agreement was observed when using the STAPLE references for the proposed DFCN cosegmentation on the PET‐CT images. The average DSCs on CT and PET are 0.861 ± 0.037 and 0.828 ± 0.087, respectively, using DFCN, compared to 0.638 ± 0.165 and 0.643 ± 0.141, respectively, when using the graph‐based cosegmentation method. The proposed DFCN cosegmentation using both PET and CT also outperforms the deep learning method using either PET or CT alone. Conclusions The proposed DFCN cosegmentation is able to outperform existing graph‐based segmentation methods. The proposed DFCN cosegmentation shows promise for further integration with quantitative multimodality imaging tools in clinical trials.

Artificial Intelligence in Nuclear Medicine

“…Zhao et al showed, for a small group of 30 patients, that the automatic segmentation of such tumors on 18 F-FDG PET/CT data was, in principle, possible using the U-Net architecture (mean Dice score of 87.47%) (44). Other groups applied similar approaches to head and neck cancer (45) and lung cancer (46,47). Still, fully automated tumor segmentation remains a challenge, probably because of the extremely diverse appearance of these diseases.…”

Radiomics: Data Are Also Images

“…DL has been especially successful in medical image segmentation tasks (7), as the learning process occurs on the voxel level and not on the entire-image level (as for classification tasks), thereby reducing the requirements regarding the amount of learning data needed for efficient training. Recently, convolutional neural network approaches were applied to PET (50) and PET/CT segmentation (53)(54)(55). DL algorithms for PET tumor detection and segmentation (56) may provide fully automated solutions for these steps of the radiomics pipeline.…”

Automatic Head and Neck Tumor Segmentation in PET/CT with Scale Attention Network

“…Andrearczyk et al [5] expanded this work by investigating several segmentation strategies based on V-Net architecture on a publicly available dataset with 202 patients. Zhao et al [10] employed a multi-modality fully convolutional network (FCN) for tumor co-segmentation in PET-CT images on a clinic dataset of 84 patients with lung cancer, and Zhong et al [11] proposed a segmentation method that consists of two coupled 3D U-Nets for simultaneously co-segmenting tumors in PET/CT images for 60 non-small cell lung cancer (NSCLC) patients.…”