Learning stochastic object models from medical imaging measurements by use of advanced ambient generative adversarial networks

Abstract

In order to objectively assess new medical imaging technologies via computer-simulations, it is important to account for all sources of variability that contribute to image data. One important source of variability that can significantly limit observer performance is associated with the variability in the ensemble of objects to-be-imaged. This source of variability can be described by stochastic object models (SOMs), which are generative models that can be employed to sample from a distribution of to-be-virtually-imaged objects. It is generally desirable to establish SOMs from experimental imaging measurements acquired by use of a well-characterized imaging system, but this task has remained challenging. Deep generative neural networks, such as generative adversarial networks (GANs) hold potential for such tasks. To establish SOMs from imaging measurements, an AmbientGAN has been proposed that augments a GAN with a measurement operator. However, the original AmbientGAN could not immediately benefit from modern training procedures and GAN architectures, which limited its ability to be applied to realistically sized medical image data. To circumvent this, in this work, a modified AmbientGAN training strategy is proposed that is suitable for modern progressive or multi-resolution training approaches such as employed in the Progressive Growing of GANs and Style-based GANs. AmbientGANs established by use of the proposed training procedure are systematically validated in a controlled way by use of computer-simulated measurement data corresponding to a stylized imaging system. Finally, emulated single-coil experimental magnetic resonance imaging data are employed to demonstrate the methods under less stylized conditions.

Mining the manifolds of deep generative models for multiple data-consistent solutions of ill-posed tomographic imaging problems

“…State-of-the-art deep generative models such as the Style-GAN [26] hold the potential for characterizing the distribution of finite-dimensional approximations of to-be-imaged objects [28], [29]. Let G : R k → R N denote a parameterized deep generative model with L layers, where k N .…”

Evaluating Procedures for Establishing Generative Adversarial Network-based Stochastic Image Models in Medical Imaging

“…Generative models, such as generative adversarial networks (GANs) are a class of models that seek to approximate unknown high-dimensional data distributions, for instance, image distributions. 1,2 GANs hold promise for potential applications in medical imaging, 3 such as unconditional medical image synthesis, 4,5 image restoration and reconstruction, [6][7][8] medical image translation 9 and data augmentation. 10 GANs have also been proposed as a tool for establishing stochastic image models (SIMs), with potential applications to objective assessment and optimization of medical imaging systems.…”

Assessing the Ability of Generative Adversarial Networks to Learn Canonical Medical Image Statistics

“…However, state-of-the-art GANs trained on medical image datasets have been shown to produce images that look realistic, but nevertheless contain potentially impactful errors [18], [23], [24]. Therefore, in order for GANs to be safely used in medical imaging applications, they must be objectively evaluated [25].…”