CT-Based Pelvic T1-Weighted MR Image Synthesis Using UNet, UNet++ and Cycle-Consistent Generative Adversarial Network (Cycle-GAN)

Kalantar, Reza; Messiou, Christina; Winfield, Jessica M.; Renn, Alexandra; Latifoltojar, Arash; Downey, Kate; Sohaib, Aslam; Lalondrelle, Susan; Koh, Dow‐Mu; Blackledge, Matthew

doi:10.3389/fonc.2021.665807

Cited by 27 publications

(18 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One group of studies focused on synthesizing MR from CT, in order to reduce time and costs involved in imaging examination (Jin et al 2019, Li et al 2020b, 2020c, Kalantar et al 2021. Two studies focused on the technical performance of methods and former one investigated the possibility of radiotherapy planning only based on CT (Jin et al 2019, Li et al 2020c, and then, one left study focused on comparing the differences of MR generation by using supervised methods and unsupervised methods (Kalantar et al 2021).…”

Section: Cross-modality I2i Translation (Cmit)mentioning

confidence: 99%

Deep learning based unpaired image-to-image translation applications for medical physics: a systematic review

Chen

Wee

et al. 2023

Phys. Med. Biol.

View full text Add to dashboard Cite

Purpose: There is a growing number of publications on the application of unpaired image-to-image (I2I) translation in medical imaging. However, a systematic review covering the current state of this topic for medical physicists is lacking. The aim of this article is to provide a comprehensive review of current challenges and opportunities for medical physicists to apply I2I translation in practice. Methods and Materials: The PubMed electronic database was searched using terms referring to unpaired (unsupervised), I2I translation, and medical imaging. This review has been reported in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. From each full-text article, we extracted information extracted regarding technical and clinical applications of methods, Transparent Reporting for Individual Prognosis Or Diagnosis (TRIPOD) study type, performance of algorithm and accessibility of source code and pre-trained models. Results: Among 461 unique records, 55 full-text articles were included in the review. The major technical applications described in the selected literature are segmentation (26 studies), unpaired domain adaptation (18 studies), and denoising (8 studies). In terms of clinical applications, unpaired I2I translation has been used for automatic contouring of regions of interest in MRI, CT, X-ray and ultrasound images, fast MRI or low dose CT imaging, CT or MRI only based radiotherapy planning, etc. Only 5 studies validated their models using an independent test set and none were externally validated by independent researchers. Finally, 12 articles published their source code and only one study published their pre-trained models. Conclusion: I2I translation of medical images offers a range of valuable applications for medical physicists. However, the scarcity of external validation studies of I2I models and the shortage of publicly available pre-trained models limits the immediate applicability of the proposed methods in practice.

show abstract

Section: Cross-modality I2i Translation (Cmit)mentioning

confidence: 99%

Deep learning based unpaired image-to-image translation applications for medical physics: a systematic review

Chen

Wee

et al. 2023

Phys. Med. Biol.

View full text Add to dashboard Cite

show abstract

“…It outperforms UNet in semantic segmentation tasks because it integrates multiple levels of features and performs more feature-splicing operations. UNet++ is similar to UNet in terms of network structure and also consists of a downsampling encoder and upsampling decoder to form a backbone network [21][22][23][24][25][26]. The UNet++ network is an improved version of the UNet network that uses a series of nested dense convolution blocks and skip connections.…”

Section: Network Structurementioning

confidence: 99%

Reading recognition of pointer meters based on an improved UNet++ network

Huo,

Bai,

Sun

et al. 2023

Meas. Sci. Technol.

View full text Add to dashboard Cite

Pointer meters are widely used in modern industries, such as petrochemical applications, substations, and nuclear power plants. To overcome the reading errors and inaccurate measurements due to uneven or fluctuating illumination in practical applications, this paper proposes an improved UNet++ network for recognizing pointer meter readings. First, the scale invariant feature transform feature-matching algorithm is used to adjust the captured tilted meter images to a symmetrical and upright shape. Then, the UNet++ network is used to segment the scale and pointer regions in the dashboard to eliminate background interference. Furthermore, part of the convolution in the UNet++ network is replaced with dilated convolution with different expansion rates to expand the perceptual field during network training. In the UNet++ network jump connection, the attention mechanism module is also introduced in the path to enhance the region’s features to be segmented and suppress the parts of the non-segmented area. A hybrid loss function is used for the network model training to prevent the imbalance of the segmented region share. Finally, the distance method is used to read the gauge representation. Experiments were conducted to compare the performance of the proposed method with that of the original UNet++ network in terms of feasibility and precision. The experimental results showed that the recognition reading accuracy was significantly improved by the enhanced network, with the accuracy, sensitivity, and specificity reaching 98.65%, 84.33%, and 99.38%, respectively. Furthermore, when using the improved UNet++ network for numerical reading, the average relative error was only 0.122%, indicating its robustness in a natural environment.

show abstract

“…The global loss function can capture image features by comparing the statistics collected over the entire image. Specifically, the perceptual loss aims to synthesize T1WI planning CTs using deep learning (DL)-based frameworks, U-Net and CycleGAN (24). The contextual loss function has been used in a fully convolutional neural network (FCN) to generate pseudo-CTs from MRI, which confirms that it can improve the predicted performance of the CNN without changing the network architecture (26).…”

Section: Introductionmentioning

confidence: 96%

“…There are two types of commonly used loss functions for optimizing the generated values of synthetic MR images: pixel-to-pixel loss function and global loss function (23). The former compares the predicted and actual values pixel-by-pixel under the same spatial coordinates to obtain characteristics such as mean square error (MSE) (17,20,24), binary cross-entropy (BCE) (25), etc. The global loss function can capture image features by comparing the statistics collected over the entire image.…”

Section: Introductionmentioning

confidence: 99%

Synthesis of magnetic resonance images from computed tomography data using convolutional neural network with contextual loss function

Li¹,

Huang²,

Zhang³

et al. 2022

Quant Imaging Med Surg

View full text Add to dashboard Cite

Background: Magnetic resonance imaging (MRI) images synthesized from computed tomography (CT) data can provide more detailed information on pathological structures than that of CT data alone; thus, the synthesis of MRI has received increased attention especially in medical scenarios where only CT images are available. A novel convolutional neural network (CNN) combined with a contextual loss function was proposed for synthesis of T1-and T2-weighted images (T1WI and T2WI) from CT data.Methods: A total of 5,053 and 5,081 slices of T1WI and T2WI, respectively were selected for the dataset of CT and MRI image pairs. Affine registration, image denoising, and contrast enhancement were done on the aforementioned multi-modality medical image dataset comprising T1WI, T2WI, and CT images of the brain. A deep CNN was then proposed by modifying the ResNet structure to constitute the encoder and decoder of U-Net, called double ResNet-U-Net (DRUNet). Three different loss functions were utilized to optimize the parameters of the proposed models: mean squared error (MSE) loss, binary crossentropy (BCE) loss, and contextual loss. Statistical analysis of the independent-sample t-test was conducted by comparing DRUNets with different loss functions and different network layers. Results: DRUNet-101 with contextual loss yielded higher values of peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and Tenengrad function (i.e., 34.25±2.06, 0.97±0.03, and 17.03±2.75 for T1WI and 33.50±1.08, 0.98±0.05, and 19.76±3.54 for T2WI respectively). The results were statistically significant at P<0.001 with a narrow confidence interval of difference, indicating the superiority of DRUNet-101 with contextual loss. In addition, both image zooming and difference maps presented for the final synthetic MR images visually reflected the robustness of DRUNet-101 with contextual loss. The visualization of convolution filters and feature maps showed that the proposed model can generate synthetic MR images with high-frequency information. Conclusions: The results demonstrated that DRUNet-101 with contextual loss function provided better high-frequency information in synthetic MR images compared with the other two functions. The proposed DRUNet model has a distinct advantage over previous models in terms of PSNR, SSIM, and Tenengrad score.Overall, DRUNet-101 with contextual loss is recommended for synthesizing MR images from CT scans.

show abstract

CT-Based Pelvic T1-Weighted MR Image Synthesis Using UNet, UNet++ and Cycle-Consistent Generative Adversarial Network (Cycle-GAN)

Cited by 27 publications

References 25 publications

Deep learning based unpaired image-to-image translation applications for medical physics: a systematic review

Deep learning based unpaired image-to-image translation applications for medical physics: a systematic review

Reading recognition of pointer meters based on an improved UNet++ network

Synthesis of magnetic resonance images from computed tomography data using convolutional neural network with contextual loss function

Contact Info

Product

Resources

About