A Unified Multi-Phase CT Synthesis and Classification Framework for Kidney Cancer Diagnosis With Incomplete Data

Abstract

Multi-phase CT is widely adopted for the diagnosis of kidney cancer due to the complementary information among phases. However, the complete set of multi-phase CT is often not available in practical clinical applications. In recent years, there have been some studies to generate the missing modality image from the available data. Nevertheless, the generated images are not guaranteed to be effective for the diagnosis task. In this paper, we propose a unified framework for kidney cancer diagnosis with incomplete multi-phase CT, which simultaneously recovers missing CT images and classifies cancer subtypes using the completed set of images. The advantage of our framework is that it encourages a synthesis model to explicitly learn to generate missing CT phases that are helpful for classifying cancer subtypes. We further incorporate lesion segmentation network into our framework to exploit lesion-level features for effective cancer classification in the whole CT volumes. The proposed framework is based on fully 3D convolutional neural networks to jointly optimize both synthesis and classification of 3D CT volumes. Extensive experiments on both in-house and external datasets demonstrate the effectiveness of our framework for the diagnosis with incomplete data compared with state-of-theart baselines. In particular, cancer subtype classification using the completed CT data by our method achieves higher performance than the classification using the given incomplete data.

Conditional Generative Learning for Medical Image Imputation

“…In Table 2, we compare the performance on the cGAN and PIX2PIX algorithms 8/19 implemented in our study with the performance of the two top-performing algorithms reported in a recent CECT renal image imputation study [32]. These algorithms are 1.…”

“…These algorithms are 1. DiagnosisGAN [32], which is a generative adversarial algorithm like the cGAN. However unlike the cGAN approach, it does not provide any information regarding the confidence in the imputation and further it requires segmented tumor images to improve its performance.…”

“…It also includes algorithms for generating CECT images at a single time point for the lungs [21], and the kidneys [27], where the latter study uses the concept of neural transfer for improved performance. Recently, several GAN-based approaches were implemented and tested for imputing renal CECT images at different time-points [32]. The approaches tested include several standard algorithms and two novel methods, ReMIC and DiagnosisGAN, which were shown to be the most accurate.…”

“…The approaches tested include several standard algorithms and two novel methods, ReMIC and DiagnosisGAN, which were shown to be the most accurate. In ReMIC [28], a representational disentanglement scheme for multi-domain image completion was used to improve the performance of the algorithm, whereas in DiagnosisGAN [32] in addition to the CECT images themselves, other sources of information, like segmentation mask for the tumor, and the knowledge cancer sub-type, were used to improve the performance of the method. In the results section of this manuscript we compare our algorithm with these methods and conclude that our method is more accurate, and at the same time provides estimates of confidence in the imputation task.…”

“…We briefly describe two existing deep generative models that have been developed for medical image imputation tasks. The first is DiagnosisGAN [32], which is a specialized GAN model that can simultaneously generate missing image in CT sequences and classify the cancer subtype. In this model, the generator take an incomplete multi-phase CT sequence as input and generates a synthesized volume for the missing phase.…”