Multichannel Residual Conditional GAN-Leveraged Abdominal Pseudo-CT Generation via Dixon MR Images

Abstract

Magnetic resonance (MR) images have distinctive advantages in radiation treatment (RT) planning due to their superior, anatomic and functional information compared with computed tomography (CT). For the RT dose calculation, MR images cannot be directly used because of the lack of electron density information. To address this issue, we propose to generate pseudo-CT (pCT) in terms of multiple matching Dixon MR images to support MR-only RT, particularly in the challenging body section of the abdomen. To this end, we design the dedicated multichannel residual conditional generative adversarial network (MCRCGAN). The significance of our efforts is threefold: 1) The MCRCGAN organically incorporates multiple theories and techniques, such as multichannel residual network (ResNet) and conditional generative adversarial network (cGAN), which facilitate its more authentic pCT generation than many existing methods. 2) The usage of residual modules effectively deepens the network without performance degradation, and the multichannel ResNet helps to simultaneously capture the substance of images, as extensively as possible, which is implicitly contained in the multiple different MR images of the same subject. 3) Due to the designed dedicated network structure, the MCRCGAN is capable of generating satisfactory pCTs under the condition of limited training data as well as prompt prediction response. Experimental studies on ten patients' paired MR-CT images demonstrate the effectiveness of our proposed MCRCGAN model on both the pCT generation quality and the performance stability. INDEX TERMS Generative adversarial network (GAN), pseudo-CT, abdomen, deep learning.

Deep learning based synthetic‐CT generation in radiotherapy and PET: A review

“…The total number of patients included in the analysis was variable, but most studies dealt with less than 50 patients for all three categories. The largest patient cohorts included 402 65 (I), 328 66 (II), and 193 patients 67 (I), while the smallest studies included 10 patients 68 and another 10 volunteers 69 (I).…”

Deep learning-based synthetic-CT generation in radiotherapy and PET: a review

“…Generally, a single MRI sequence is used as input. However, eight studies investigated using multiple input sequences or Dixon reconstructions 63,66,80,88,89,92,102,115 based on the assumption that more input contrast may facilitate sCT generation. Some studies compared the performance of sCT generation depending on the sequence acquired.…”

Attribute-Aware Generative Design With Generative Adversarial Networks

“…Since its introduction in 2014, GAN [17] continues to attract growing interests in the deep learning community and has been applied to various domains such as computer vision [28]- [33], natural language processing [34], [35], time series synthesis [36], [37], and semantic segmentation [38], [39]. Specifically, GAN has shown significant recent success in the field of computer vision on a variety of tasks such as image generation [28], [29], image to image translation [30], [31], and image super-resolution [32], [33]. The standard GAN structure comprises two neural networks: a generator G and a discriminator D which are iteratively trained by competing against each other in a minimax game, where the generator attempts to produce realistic samples while the discriminator attempts to distinguish the fake samples from the real ones.…”