Single‐image Tomography: 3D Volumes from 2D Cranial X‐Rays

Henzler, Philipp; Rasche, Volker; Ropinski, Timo; Ritschel, Tobias

doi:10.1111/cgf.13369

Cited by 98 publications

(75 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We evaluate the proposed X2CT-GAN model with several widely used metrics, e.g., peak signalto-noise ratio (PSNR) and structural similarity (SSIM) index. To demonstrate the effectiveness of our method, we reproduce a baseline model named 2DCNN [13]. Fair comparisons and comprehensive analysis are given to demonstrate the improvement of our proposed method over the baseline and other mutants.…”

Section: Methodsmentioning

confidence: 99%

“…Finally, we show the real-world X-ray evaluation results of X2CT-GAN. Input images to X2CT-GAN are resized to 128 × 128 pixels, while the input of 2DCNN is 256 × 256 pixels as suggested by [13]. The output of all models is set to 128 × 128 × 128 voxels.…”

Section: Methodsmentioning

confidence: 99%

“…And, [12] mainly deals with the limitedangle CT compensation problem. More relevant to our work is [13], which uses a convolutional neural network (CNN) to predict the underlying 3D object as a volume from a singleimage tomography. However, we argue that a single X-ray is not enough to accurately reconstruct 3D anatomy since it is subject to too much ambiguity.…”

Section: Related Workmentioning

confidence: 99%

“…As shown in our experiments, biplanar X-rays with two orthogonal projections can significantly improve the reconstruction accuracy, benefiting from more constraints provided by an additional view. Furthermore, the images reconstructed by [13] are quite blurry, thus with limited clinical values. Combining adversarial training and reconstruction constraints, our method could extract much finer anatomical structures (e.g.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

X2CT-GAN: Reconstructing CT From Biplanar X-Rays With Generative Adversarial Networks

Ying

Guo

Ma³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

180

119

View full text Add to dashboard Cite

Computed tomography (CT) can provide a 3D view of the patient's internal organs, facilitating disease diagnosis, but it incurs more radiation dose to a patient and a CT scanner is much more cost prohibitive than an X-ray machine too. Traditional CT reconstruction methods require hundreds of X-ray projections through a full rotational scan of the body, which cannot be performed on a typical X-ray machine. In this work, we propose to reconstruct CT from two orthogonal X-rays using the generative adversarial network (GAN) framework. A specially designed generator network is exploited to increase data dimension from 2D (X-rays) to 3D (CT), which is not addressed in previous research of GAN. A novel feature fusion method is proposed to combine information from two X-rays. The mean squared error (MSE) loss and adversarial loss are combined to train the generator, resulting in a high-quality CT volume both visually and quantitatively. Extensive experiments on a publicly available chest CT dataset demonstrate the effectiveness of the proposed method. It could be a nice enhancement of a low-cost X-ray machine to provide physicians a CT-like 3D volume in several niche applications.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

X2CT-GAN: Reconstructing CT From Biplanar X-Rays With Generative Adversarial Networks

Ying

Guo

Ma³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

180

119

View full text Add to dashboard Cite

show abstract

“…To alleviate this issue, Liu et al [33] tried a wavelet-based reconstruction approach to the acquired singe-view measurements, but the reconstruction quality is still not satisfactory for clinical applications. Recently, Henzler et al [19] proposed a convolutional encoder-decoder network to reconstruct a 3D volume from a 2D single-view cranial X-ray image. The direct coarse output is then improved to higher resolution by post fusion.…”

Section: Volumetric Image Reconstruction Methodsmentioning

confidence: 99%

DeepOrganNet: On-the-Fly Reconstruction and Visualization of 3D / 4D Lung Models from Single-View Projections by Deep Deformation Network

Wang

Zhong

Hua

2019

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

This paper introduces a deep neural network based method, i.e., DeepOrganNet, to generate and visualize fully high-fidelity 3D / 4D organ geometric models from single-view medical images with complicated background in real time. Traditional 3D / 4D medical image reconstruction requires near hundreds of projections, which cost insufferable computational time and deliver undesirable high imaging / radiation dose to human subjects. Moreover, it always needs further notorious processes to segment or extract the accurate 3D organ models subsequently. The computational time and imaging dose can be reduced by decreasing the number of projections, but the reconstructed image quality is degraded accordingly. To our knowledge, there is no method directly and explicitly reconstructing multiple 3D organ meshes from a single 2D medical grayscale image on the fly. Given single-view 2D medical images, e.g., 3D / 4D-CT projections or X-ray images, our end-to-end DeepOrganNet framework can efficiently and effectively reconstruct 3D / 4D lung models with a variety of geometric shapes by learning the smooth deformation fields from multiple templates based on a trivariate tensor-product deformation technique, leveraging an informative latent descriptor extracted from input 2D images. The proposed method can guarantee to generate high-quality and high-fidelity manifold meshes for 3D / 4D lung models; while, all current deep learning based approaches on the shape reconstruction from a single image cannot. The major contributions of this work are to accurately reconstruct the 3D organ shapes from 2D single-view projection, significantly improve the procedure time to allow on-the-fly visualization, and dramatically reduce the imaging dose for human subjects. Experimental results are evaluated and compared with the traditional reconstruction method and the state-of-the-art in deep learning, by using extensive 3D and 4D examples, including both synthetic phantom and real patient datasets. The efficiency of the proposed method shows that it only needs several milliseconds to generate organ meshes with 10K vertices, which has a great potential to be used in real-time image guided radiation therapy (IGRT).

show abstract

Three‐dimensional image volumes from two‐dimensional digitally reconstructed radiographs: A deep learning approach in lower limb CT scans

2021

View full text Add to dashboard Cite

Purpose: Three-dimensional (3D) reconstructions of the human anatomy have been available for surgery planning or diagnostic purposes for a few years now. The different image modalities usually rely on several consecutive two-dimensional (2D) acquisitions in order to reconstruct the 3D volume. Hence, such acquisitions are expensive, time-demanding and often expose the patient to an undesirable amount of radiation. For such reasons, along the most recent years, several studies have been proposed that extrapolate 3D anatomical features from merely 2D exams such as x rays for implant templating in total knee or hip arthroplasties. Method: The presented study shows an adaptation of a deep learning-based convolutional neural network to reconstruct 3D volumes from a mere 2D digitally reconstructed radiograph from one of the most extensive lower limb computed tomography datasets available. This novel approach is based on an encoder-decoder architecture with skip connections and a multidimensional Gaussian filter as data augmentation technique. Results: The results achieved promising values when compared against the ground truth volumes, quantitatively yielding an average of 0.77 AE 0.05 structured similarity index. Conclusions: A novel deep learning-based approach to reconstruct 3D medical image volumes from a single x-ray image was shown in the present study. The network architecture was validated against the original scans presenting SSIM values of 0.77 AE 0.05 and 0.78 AE 0.06, respectively for the knee and the hip crop.

show abstract

Single‐image Tomography: 3D Volumes from 2D Cranial X‐Rays

Cited by 98 publications

References 38 publications

X2CT-GAN: Reconstructing CT From Biplanar X-Rays With Generative Adversarial Networks

X2CT-GAN: Reconstructing CT From Biplanar X-Rays With Generative Adversarial Networks

DeepOrganNet: On-the-Fly Reconstruction and Visualization of 3D / 4D Lung Models from Single-View Projections by Deep Deformation Network

Three‐dimensional image volumes from two‐dimensional digitally reconstructed radiographs: A deep learning approach in lower limb CT scans

Contact Info

Product

Resources

About