Unpaired Mr to CT Synthesis with Explicit Structural Constrained Adversarial Learning

Ge, Yunhao; Wei, Dongming; Zhang, Xue; Zhou, Xiang; Zhan, Yiqiang; Liao, Shu

doi:10.1109/isbi.2019.8759529

Cited by 31 publications

(24 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However,

L_{c o n v e n t i o n a l}

can cause malformed prediction images because the similarities between the real and synthesized images are added based on the

L_{c y c}

. To solve this problem, the proposed algorithm includes mutual information (

L_{M I}

) and shape consistency losses (

L_{S C}

) 46,47 . Mutual information is a measure of the information redundancy of the image intensities between different modality images and is defined as follows:

L_{M I} = \sum_{x \in G_{k V} false(I_{M V} false)} \sum_{y \in I_{M V}} p false(x, y false) \log (\frac{p false(x, y false)}{p false(x false) p false(y false)}),

where

p (x)

and

p (y)

denote the marginal distributions of

G_{k V} false(I_{M V} false)

and

I_{M V}

, respectively; and

p (x, y)

denotes the joint distributions of

G_{k V} false(I_{M V} false)

and

I_{M V}

53 .…”

Section: Methodsmentioning

confidence: 99%

Improvement of megavoltage computed tomography image quality for adaptive helical tomotherapy using cycleGAN‐based image synthesis with small datasets

et al. 2021

View full text Add to dashboard Cite

Megavoltage computed tomography (MVCT) offers an opportunity for adaptive helical tomotherapy. However, high noise and reduced contrast in the MVCT images due to a decrease in the imaging dose to patients limits its usability. Therefore, we propose an algorithm to improve the image quality of MVCT. Methods:The proposed algorithm generates kilovoltage CT (kVCT)-like images from MVCT images using a cycle-consistency generative adversarial network (cycleGAN)-based image synthesis model. Data augmentation using an affine transformation was applied to the training data to overcome the lack of data diversity in the network training. The mean absolute error (MAE), root-meansquare error (RMSE), peak signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM) were used to quantify the correction accuracy of the images generated by the proposed algorithm. The proposed method was validated by comparing the images generated with those obtained from conventional and deep learning-based image processing method through nonaugmented datasets. Results: The average MAE, RMSE, PSNR, and SSIM values were 18.91 HU, 69.35 HU, 32.73 dB, and 95.48 using the proposed method, respectively, whereas cycleGAN with non-augmented data showed inferior results (19.88 HU, 70.55 HU, 32.62 dB, 95.19, respectively). The voxel values of the image obtained by the proposed method also indicated similar distributions to those of the kVCT image. The dose-volume histogram of the proposed method was also similar to that of electron density corrected MVCT. Conclusions: The proposed algorithm generates synthetic kVCT images from MVCT images using cycleGAN with small patient datasets. The image quality achieved by the proposed method was correspondingly improved to the level of a kVCT image while maintaining the anatomical structure of an MVCT image. The evaluation of dosimetric effectiveness of the proposed method indicates the applicability of accurate treatment planning in adaptive radiation therapy.

show abstract

“…However,

L_{c o n v e n t i o n a l}

can cause malformed prediction images because the similarities between the real and synthesized images are added based on the

L_{c y c}

. To solve this problem, the proposed algorithm includes mutual information (

L_{M I}

) and shape consistency losses (

L_{S C}

) 46,47 . Mutual information is a measure of the information redundancy of the image intensities between different modality images and is defined as follows:

L_{M I} = \sum_{x \in G_{k V} false(I_{M V} false)} \sum_{y \in I_{M V}} p false(x, y false) \log (\frac{p false(x, y false)}{p false(x false) p false(y false)}),

where

p (x)

and

p (y)

denote the marginal distributions of

G_{k V} false(I_{M V} false)

and

I_{M V}

, respectively; and

p (x, y)

denotes the joint distributions of

G_{k V} false(I_{M V} false)

and

I_{M V}

53 .…”

Section: Methodsmentioning

confidence: 99%

Improvement of megavoltage computed tomography image quality for adaptive helical tomotherapy using cycleGAN‐based image synthesis with small datasets

et al. 2021

View full text Add to dashboard Cite

show abstract

“…A major obstacle in whole-body PET/MRI AC investigations is the limited availability of a large number of registered CT and MRI pairs and the lack of accuracy of nonlinear registration between them. A promising approach to overcome this challenge is the use of CycleGAN that employs cycle consistency loss as an indirect structural similarity between the input and synthesized images [223][224][225]. Further evaluation of this method for PET/MRI AC will be necessary.…”

Section: Deep Learning: Future Directionmentioning

confidence: 99%

A Review of Deep-Learning-Based Approaches for Attenuation Correction in Positron Emission Tomography

Lee

2021

IEEE Trans. Radiat. Plasma Med. Sci.

View full text Add to dashboard Cite

Attenuation correction (AC) is essential for the generation of artifact-free and quantitaAtively accurate positron emission tomography (PET) images. PET AC based on computed tomography (CT) frequently results in artifacts in attenuationcorrected PET images, and these artifacts mainly originate from CT artifacts and PET-CT mismatches. The AC in PET combined with a magnetic resonance imaging (MRI) scanner (PET/MRI) is more complex than PET/CT, given that MR images do not provide direct information on high energy photon attenuation. Deep learning (DL)-based methods for the improvement of PET AC have received significant research attention as alternatives to conventional AC methods. Many DL studies were focused on the transformation of MR images into synthetic pseudo-CT or attenuation maps. Alternative approaches that are not dependent on the anatomical images (CT or MRI) can overcome the limitations related to current CT-and MRI-based ACs and allow for more accurate PET quantification in stand-alone PET scanners for the realization of low radiation doses. In this article, a review is presented on the limitations of the PET AC in current dual-modality PET/CT and PET/MRI scanners, in addition to the current status and progress of DL-based approaches, for the realization of improved performance of PET AC.

show abstract

“…However, MR image does not provide electron density information that computed tomography (CT) image can provide, which is essential for applications like dose calculation in radiotherapy treatment planning [2,10,5,18] and attenuation correction in positron emission tomography reconstruction [24,22,16]. To overcome this limitation, a variety of approaches have been proposed to recreate a CT image from the available MR images [25,11,26,31,9]. Recently, deep learning-based synthesis methods [25,11,26,31,9,19] have shown superior performance over alternatives such as segmentation-based [17,1,5] and atlas-based methods [30,3,7,4].…”

Section: Introductionmentioning

confidence: 99%

“…To overcome this limitation, a variety of approaches have been proposed to recreate a CT image from the available MR images [25,11,26,31,9]. Recently, deep learning-based synthesis methods [25,11,26,31,9,19] have shown superior performance over alternatives such as segmentation-based [17,1,5] and atlas-based methods [30,3,7,4].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Frequency-Supervised MR-to-CT Image Synthesis

Shi¹,

Mettes²,

Zheng³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper strives to generate a synthetic computed tomography (CT) image from a magnetic resonance (MR) image. The synthetic CT image is valuable for radiotherapy planning when only an MR image is available. Recent approaches have made large strides in solving this challenging synthesis problem with convolutional neural networks that learn a mapping from MR inputs to CT outputs. In this paper, we find that all existing approaches share a common limitation: reconstruction breaks down in and around the high-frequency parts of CT images. To address this common limitation, we introduce frequencysupervised deep networks to explicitly enhance high-frequency MR-to-CT image reconstruction. We propose a frequency decomposition layer that learns to decompose predicted CT outputs into low-and highfrequency components, and we introduce a refinement module to improve high-frequency reconstruction through high-frequency adversarial learning. Experimental results on a new dataset with 45 pairs of 3D MR-CT brain images show the effectiveness and potential of the proposed approach. Code is available at https://github.com/shizenglin/ Frequency-Supervised-MR-to-CT-Image-Synthesis.

show abstract

Unpaired Mr to CT Synthesis with Explicit Structural Constrained Adversarial Learning

Cited by 31 publications

References 6 publications

Improvement of megavoltage computed tomography image quality for adaptive helical tomotherapy using cycleGAN‐based image synthesis with small datasets

Improvement of megavoltage computed tomography image quality for adaptive helical tomotherapy using cycleGAN‐based image synthesis with small datasets

A Review of Deep-Learning-Based Approaches for Attenuation Correction in Positron Emission Tomography

Frequency-Supervised MR-to-CT Image Synthesis

Contact Info

Product

Resources

About