Deep learning approaches using 2D and 3D convolutional neural networks for generating male pelvic synthetic computed tomography from magnetic resonance imaging

Abstract

Purpose The improved soft tissue contrast of magnetic resonance imaging (MRI) compared to computed tomography (CT) makes it a useful imaging modality for radiotherapy treatment planning. Even when MR images are acquired for treatment planning, the standard clinical practice currently also requires a CT for dose calculation and x‐ray–based patient positioning. This increases workloads, introduces uncertainty due to the required inter‐modality image registrations, and involves unnecessary irradiation. While it would be beneficial to use exclusively MR images, a method needs to be employed to estimate a synthetic CT (sCT) for generating electron density maps and patient positioning reference images. We investigated 2D and 3D convolutional neural networks (CNNs) to generate a male pelvic sCT using a T1‐weighted MR image and compare their performance. Methods A retrospective study was performed using CTs and T1‐weighted MR images of 20 prostate cancer patients. CTs were deformably registered to MR images to create CT‐MR pairs for training networks. The proposed 2D CNN, which contained 27 convolutional layers, was modified from the state‐of‐the‐art 2D CNN to save computational memory and prepare for building the 3D CNN. The proposed 2D and 3D models were trained from scratch to map intensities of T1‐weighted MR images to CT Hounsfield Unit (HU) values. Each sCT was generated in a fivefold cross‐validation framework and compared with the corresponding deformed CT (dCT) using voxel‐wise mean absolute error (MAE). The sCT geometric accuracy was evaluated by comparing bone regions, defined by thresholding at 150 HU in the dCTs and the sCTs, using dice similarity coefficient (DSC), recall, and precision. To evaluate sCT patient positioning accuracy, bone regions in dCTs and sCTs were rigidly registered to the corresponding cone‐beam CTs. The resulting paired Euler transformation vectors were compared by calculating translation vector distances and absolute differences of Euler angles. Statistical tests were performed to evaluate the differences among the proposed models and Han’s model. Results Generating a pelvic sCT required approximately 5.5 s using the proposed models. The average MAEs within the body contour were 40.5 ± 5.4 HU (mean ± SD) and 37.6 ± 5.1 HU for the 2D and 3D CNNs, respectively. The average DSC, recall, and precision for the bone region (thresholding the CT at 150 HU) were 0.81 ± 0.04, 0.85 ± 0.04, and 0.77 ± 0.09 for the 2D CNN, and 0.82 ± 0.04, 0.84 ± 0.04, and 0.80 ± 0.08 for the 3D CNN, respectively. For both models, mean translation vector distances are less than 0.6 mm with mean absolute differences of Euler angles less than 0.5°. Conclusions The 2D and 3D CNNs generated accurate pelvic sCTs for the 20 patients using T1‐weighted MR images. Statistical tests indicated that the proposed 3D model was able to generate sCTs with smaller MAE and higher bone region precision compared to 2D models. Results of patient alignment tests suggested that sCTs generated by the proposed CNNs can provide accurate pa...

A new deep convolutional neural network design with efficient learning capability: Application to CT image synthesis from MRI

“…The proposed eCNN exhibited superior performance to the atlas-based method achieving a MAE of 30.0 AE 10.4 HU and ME of 2.8 AE 10.3 HU for the entire pelvis region while the atlasbased method resulted in a MAE and ME of 64.6 AE 21.2 HU and −0.8 AE 35.4 HU, respectively. Fu et al 49 proposed a similar model to the work of Han 28 where the batch normalization and upsampling layers were replaced with the instance norm and deconvolutional layers. The modified model resulted in MAEs of 40.5 AE 5.4 HU, 28.9 AE 4.7 HU and 159.7 AE 22.5 HU for the whole pelvis, soft-tissue, and bone, respectively.…”

“…Considering bone extraction accuracy, Fu et al reported a DSC of 0.81 AE 0.04 for bone segmented using an intensity threshold of 150 HU. 49 To facilitate the comparison, the evaluation of bone extraction was repeated using the same intensity threshold where the eCNN model resulted in a DSC of 0.84 AE 0.07, while the original U-Net model led to DSC of 0.71 AE 0.06. Overall, the eCNN method outperformed the atlas-based and original U-Net methods in terms of CT value estimation and tissue delineation.…”

Generation of abdominal synthetic CTs from 0.35T MR images using generative adversarial networks for MR-only liver radiotherapy

“…A study reported that the proposed 2D CNN generated the most accurate pelvic sCT images compared to four atlas-based methods (Arabi et al 2018). Fu et al (2019) proposed a 3D CNN that generated more accurate pelvic sCT images than Han's 2D CNN. Generative adversarial networks (GANs) were shown to have better performance in image-to-image translation tasks compared to the corresponding CNNs (Isola et al 2016, Zhu et al 2017.…”

A novel anthropomorphic multimodality phantom for MRI‐based radiotherapy quality assurance testing

“…There are sCT algorithms that have been developed for the male pelvis site that use atlas-based, statistical decomposition algorithmic and, voxel-based deep-and machine-learning methods. 16,30,[44][45][46][47] In this study we tested a single commercially available statistical decomposition algorithm-based sCT generator that takes a T 2 -weighted MRI input image. Further testing is required to evaluate the phantoms performance with other sCT methods.…”