On the Effectiveness of Least Squares Generative Adversarial Networks

Purpose Cone‐beam computed tomography (CBCT) offers advantages over conventional fan‐beam CT in that it requires a shorter time and less exposure to obtain images. However, CBCT images suffer from low soft‐tissue contrast, noise, and artifacts compared to conventional fan‐beam CT images. Therefore, it is essential to improve the image quality of CBCT. Methods In this paper, we propose a synthetic approach to translate CBCT images with deep neural networks. Our method requires only unpaired and unaligned CBCT images and planning fan‐beam CT (PlanCT) images for training. The CBCT images and PlanCT images may be obtained from other patients as long as they are acquired with the same scanner settings. Once trained, three‐dimensionally reconstructed CBCT images can be directly translated into high‐quality PlanCT‐like images. Results We demonstrate the effectiveness of our method with images obtained from 20 prostate patients, and provide a statistical and visual comparison. The image quality of the translated images shows substantial improvement in voxel values, spatial uniformity, and artifact suppression compared to those of the original CBCT. The anatomical structures of the original CBCT images were also well preserved in the translated images. Conclusions Our method produces visually PlanCT‐like images from CBCT images while preserving anatomical structures.

show abstract

“…Let

{false‖ v false‖}_{p}

denote the

L^{p}

‐norm of a vector v . Our loss function for the discriminators is based on LSGAN .

\begin{matrix} {Loss}_{D} = & λ_{D} \sum_{x \in CB} (false‖ D_{P} (G_{C \to P} false(x false)) {- bold0 false‖}_{2} + {false‖ D_{C} (x) - bold1 false‖}_{2}) \\ + λ_{D} \sum_{y \in Plan} (false‖ D_{C} (G_{P \to C} false(y false)) {- bold0 false‖}_{2} + {false‖ D_{P} (y) - bold1 false‖}_{2}) . \end{matrix}

The loss function forces

D_{P}

(and, respectively,

D_{C}

) to be trained to distinguish the distributions of real PlanCT (and, respectively, CBCT) images from synthesized ones.…”

Section: Methodsmentioning

confidence: 99%

Visual enhancement of Cone‐beam CT by use of CycleGAN

et al. 2020

View full text Add to dashboard Cite

show abstract

“…By maximizing the mutual information between the reconstruction and z, we prevented posterior collapse and constrained the decoder g(·) to effectively use the modality factors. L ADV is the adversarial loss of a Least-Squares GAN [11], used to discriminate ground truth from predicted segmentations in the unsupervised setting. L T R is the loss associated to the self-supervised signal, computed as the differentiable Dice loss betweens t+dt and s t+dt .…”

Section: Cost Function and Trainingmentioning

confidence: 99%

Temporal Consistency Objectives Regularize the Learning of Disentangled Representations

Valvano

Chartsias

Leo

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

There has been an increasing focus in learning interpretable feature representations, particularly in applications such as medical image analysis that require explainability, whilst relying less on annotated data (since annotations can be tedious and costly). Here we build on recent innovations in style-content representations to learn anatomy, imaging characteristics (appearance) and temporal correlations. By introducing a self-supervised objective of predicting future cardiac phases we improve disentanglement. We propose a temporal transformer architecture that given an image conditioned on phase difference, it predicts a future frame. This forces the anatomical decomposition to be consistent with the temporal cardiac contraction in cine MRI and to have semantic meaning with less need for annotations. We demonstrate that using this regularization, we achieve competitive results and improve semisupervised segmentation, especially when very few labelled data are available. Specifically, we show Dice increase of up to 19% and 7% compared to supervised and semi-supervised approaches respectively on the ACDC dataset. Code is available at: https://github.com/gvalvano/sdtnet.

show abstract

“…In general, the resulting DSMs from our cGAN model revealed the same or slightly worse results compared even to the normal stereo DSM. The reason for this lies in the fact that the cGAN model uses the sigmoid cross entropy loss function for the discriminator [33] that leads to the vanishing gradient problem when updating the generator using the created samples that are on the correct side of the decision boundary but are still far from the real data [49]. As a result, the discriminator believes that the created images come from real samples and causes almost no error by updating the generator as the images are on the correct side of the decision boundary.…”

Section: Cgan Vs Clsganmentioning

confidence: 99%

“…The results obtained by the cLSGAN model on both datasets quantitatively outperforms the stereo DSM and the DSMs generated by the cGAN model as they are much smoother, able to reconstruct even small parts of buildings, and do not contain any artifacts. Mao et al [49] explained that as a powerful feature of the least squares loss function that moves the created samples toward the decision boundary and penalizes samples that lie in a long way on the correct side of the decision boundary. As a result, this allows the generation of more realistic samples.…”

Section: Cgan Vs Clsganmentioning

confidence: 99%

DSM-to-LoD2: Spaceborne Stereo Digital Surface Model Refinement

et al. 2018

View full text Add to dashboard Cite

A digital surface model (DSM) provides the geometry and structure of an urban environment with buildings being the most prominent objects in it. Built-up areas change with time due to the rapid expansion of cities. New buildings are being built, existing ones are expanded, and old buildings are torn down. As a result, 3D surface models can increase the understanding and explanation of complex urban scenarios. They are very useful in numerous fields of remote sensing applications, in tasks related to 3D reconstruction and city modeling, planning, visualization, disaster management, navigation, and decision-making, among others. DSMs are typically derived from various acquisition techniques, like photogrammetry, laser scanning, or synthetic aperture radar (SAR). The generation of DSMs from very high resolution optical stereo satellite imagery leads to high resolution DSMs which often suffer from mismatches, missing values, or blunders, resulting in coarse building shape representation. To overcome these problems, we propose a method for 3D surface model generation with refined building shapes to level of detail (LoD) 2 from stereo half-meter resolution satellite DSMs using deep learning techniques. Mainly, we train a conditional generative adversarial network (cGAN) with an objective function based on least square residuals to generate an accurate LoD2-like DSM with enhanced 3D object shapes directly from the noisy stereo DSM input. In addition, to achieve close to LoD2 shapes of buildings, we introduce a new approach to generate an artificial DSM with accurate and realistic building geometries from city geography markup language (CityGML) data, on which we later perform a training of the proposed cGAN architecture. The experimental results demonstrate the strong potential to create large-scale remote sensing elevation models where the buildings exhibit better-quality shapes and roof forms than just using the matching process. Moreover, the developed model is successfully applied to a different city that is unseen during the training to show its generalization capacity.2 of 20 building shapes, including the recovery of disturbed boundaries and robust reconstruction of precise rooftop geometries, is in demand.Remote sensing technology provides several ways to measure the 3D urban morphology. Conventional ground surveying, stereo airborne or satellite photogrammetry, interferometric synthetic aperture radar (InSAR), and light detection and ranging (LIDAR) are the main data sources used to obtain high-resolution elevation information [1]. The main advantage of digital surface models (DSMs) generated using ground surveying and LIDAR is their good quality and detailed object representations. However, their production is costly and time consuming, and covers relatively small areas compared with images produced with spaceborne remote sensing [2]. SAR imagery is operational in all seasons under different weather conditions. Nevertheless it has a side-looking sensor principle that is not so useful for building recognition a...

show abstract

On the Effectiveness of Least Squares Generative Adversarial Networks

Cited by 172 publications

References 30 publications

Visual enhancement of Cone‐beam CT by use of CycleGAN

Visual enhancement of Cone‐beam CT by use of CycleGAN

Temporal Consistency Objectives Regularize the Learning of Disentangled Representations

DSM-to-LoD2: Spaceborne Stereo Digital Surface Model Refinement

Contact Info

Product

Resources

About