Generative Adversarial Network (GAN) for Automatic Reconstruction of the 3D Spine Structure by Using Simulated Bi-Planar X-ray Images

Abstract

In this study, we modified the previously proposed X2CT-GAN to build a 2Dto3D-GAN of the spine. This study also incorporated the radiologist’s perspective in the adjustment of input signals to prove the feasibility of the automatic production of three-dimensional (3D) structures of the spine from simulated bi-planar two-dimensional (2D) X-ray images. Data from 1012 computed tomography (CT) studies of 984 patients were retrospectively collected. We tested this model under different dataset sizes (333, 666, and 1012) with different bone signal conditions to observe the training performance. A 10-fold cross-validation and five metrics—Dice similarity coefficient (DSC) value, Jaccard similarity coefficient (JSC), overlap volume (OV), and structural similarity index (SSIM)—were applied for model evaluation. The optimal mean values for DSC, JSC, OV, SSIM_anteroposterior (AP), and SSIM_Lateral (Lat) were 0.8192, 0.6984, 0.8624, 0.9261, and 0.9242, respectively. There was a significant improvement in the training performance under empirically enhanced bone signal conditions and with increasing training dataset sizes. These results demonstrate the potential of the clinical implantation of GAN for automatic production of 3D spine images from 2D images. This prototype model can serve as a foundation in future studies applying transfer learning for the development of advanced medical diagnostic techniques.

Automatic 3D reconstruction of vertebrae from orthogonal bi-planar radiographs

“…Best results are marked bold. VerSe-full VerSe-small Dice (+) HD95 (−) NSD (+) Dice (+) HD95 (−) NSD (+) TransVert 1 0.7743 5.4480 0.6718 0.7647 6.6468 0.6094 BX2S-Net 2 0.7558 6.2984 0.6345 0.7494 7.5000 0.5824 Unet 3 0.7712 5.5935 0.6691 0.7582 6.7097 0.6160 SwinUNETR 4 0.7759 5.2873 0.6821 0.7531 6.9923 0.5964 UNETR 5 0.7584 5.9160 0.6546 0.7421 6.7891 0.6012 AttentionUnet 6 0.7696 5.6788 0.6660 0.7535 6.8888 0.6085 OneDConcat 7 0.7822 5.0832 0.6904 0.7637 6.6105 …”

Enhancing X-ray-Based Wrist Fracture Diagnosis Using HyperColumn-Convolutional Block Attention Module

“…Meanwhile, Moon et al achieved a mean average precision (mAP) of 69.8% across diverse facial fracture datasets using the YoloX-S object-detection model with CT images, which inherently provide richer information compared to X-rays [ 27 ]. Venturing into the 3D imaging space, Yang et al utilized generative adversarial networks (GANs) to extrapolate 3D spinal structures from 2D X-ray images, leveraging a dataset of 1012 CT images from 984 patients [ 28 ].…”

Synthetic 3D Spinal Vertebrae Reconstruction from Biplanar X-rays Utilizing Generative Adversarial Networks

“…In this study, we applied the GAN-based framework for reconstructing 3D spinal vertebrae structures from synthetic biplanar X-ray images as presented by Ying et al (X2CT-GAN), focusing specifically on anterior and lateral views of spinal vertebrae [11]. In contrast to previous work [11,12] which applied X2CT-GAN for 3D reconstruction in medical settings, we focussed on segmented vertebrae to specifically focus on the region of interest (spinal vertebrae) while reducing unnecessary information and thus computational cost. This approach leverages a novel feature fusion technique based on X2CT-GAN to combine information from both views and employs a combination of mean squared error (MSE) loss and adversarial loss to train the generator, resulting in high-quality synthetic 3D spinal vertebrae CTs.…”