Impact of deep learning-based image super-resolution on binary signal detection

Zhang, Xiaohui; Kelkar, Varun A.; Granstedt, Jason L.; Li, Hua; Anastasio, Mark A.

doi:10.1117/1.jmi.8.6.065501

Cited by 22 publications

(22 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…18 Lastly, we emphasize that a task-based assessment of image reconstruction methods is needed in order to translate these methods into the clinic. 1,19,20…”

Section: Discussionmentioning

confidence: 99%

Prior image-based medical image reconstruction using a style-based generative adversarial network

Kelkar¹,

Anastasio²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Computed medical imaging systems require a computational reconstruction procedure for image formation. In order to recover a useful estimate of the object to-be-imaged when the recorded measurements are incomplete, prior knowledge about the nature of object must be utilized. In order to improve the conditioning of an ill-posed imaging inverse problem, deep learning approaches are being actively investigated for better representing object priors and constraints. This work proposes to use a style-based generative adversarial network (StyleGAN) to constrain an image reconstruction problem in the case where additional information in the form of a prior image of the sought-after object is available. An optimization problem is formulated in the intermediate latent-space of a StyleGAN, that is disentangled with respect to meaningful image attributes or "styles", such as the contrast used in magnetic resonance imaging (MRI). Discrepancy between the sought-after and prior images is measured in the disentangled latent-space, and is used to regularize the inverse problem in the form of constraints on specific styles of the disentangled latent-space. A stylized numerical study inspired by MR imaging is designed, where the sought-after and the prior image are structurally similar, but belong to different contrast mechanisms. The presented numerical studies demonstrate the superiority of the proposed approach as compared to classical approaches in the form of traditional metrics.

show abstract

“…18 Lastly, we emphasize that a task-based assessment of image reconstruction methods is needed in order to translate these methods into the clinic. 1,19,20…”

Section: Discussionmentioning

confidence: 99%

Prior image-based medical image reconstruction using a style-based generative adversarial network

Kelkar¹,

Anastasio²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The CNN-approximated IO (CNN-IO) has been successfully applied to reconstructed images in several studies. 3,[11][12][13] However, considering that diagnostically significant features are differently represented in measurement space when compared with those in image space, the effectiveness of the CNN-IO when applied to raw imaging measurements remains uninvestigated.…”

Section: The Ideal Observer (Io) and The Cnn-approximated Iomentioning

confidence: 99%

Estimating task-based performance bounds for image reconstruction methods by use of learned-ideal observers

Zhou

et al. 2023

Medical Imaging 2023: Image Perception, Observer Performance, and Technology Assessment

Self Cite

View full text Add to dashboard Cite

Medical imaging systems are commonly assessed and optimized by use of objective measures of image quality (IQ). The performance of the Ideal Observer (IO) acting on imaging measurements has long been advocated as a figureof-merit to guide the optimization of imaging systems. For computed imaging systems, the performance of the IO acting on imaging measurements also sets an upper bound on task-performance that no image reconstruction method can transcend. As such, estimation of IO performance can provide valuable guidance when designing data-acquisition techniques by enabling the identification of designs that will not permit the reconstruction of diagnostically useful images for a specified task -no matter how advanced the reconstruction method is or plausible the reconstructed images appear. The need for such analyses is urgent because of the ubiquitous development of deep learning-based image reconstruction methods and the fact that they are often not assessed by use of objective image quality measures. However, until recently, estimation of the IO was generally intractable when clinically relevant objects and imaging conditions were assumed. In this work, for the first time, estimates of the IO acting on tomographic imaging measurements were computed with consideration of realistic object variability to establish task-based performance bounds for image reconstruction methods. This was accomplished by use of a recently developed learning-based procedure. Numerical studies that were inspired by breast Xray computed tomography were conducted to validate and demonstrate the approach. The effectiveness of the approximation method was validated on raw measurements for a signal-known-exactly and backgroundknown-exactly (SKE/BKE) binary signal detection task. A signal-known-statistically and background-knownstatistically (SKS/BKS) binary signal detection task was also addressed and the impact of the number of views on IO performance was assessed.

show abstract

“…10 GANs have also been proposed as a tool for establishing stochastic image models (SIMs), with potential applications to objective assessment and optimization of medical imaging systems. [11][12][13][14] Modern GANs such as the StyleGANs 15,16 are known to produce diverse, visually realistic images.…”

Section: Purposementioning

confidence: 99%

Evaluating Procedures for Establishing Generative Adversarial Network-based Stochastic Image Models in Medical Imaging

Kelkar,

Gotsis,

Brooks

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Modern generative models, such as generative adversarial networks (GANs), hold tremendous promise for several areas of medical imaging, such as unconditional medical image synthesis, image restoration, reconstruction and translation, and optimization of imaging systems. However, procedures for establishing stochastic image models (SIMs) using GANs remain generic and do not address specific issues relevant to medical imaging. In this work, canonical SIMs that simulate realistic vessels in angiography images are employed to evaluate procedures for establishing SIMs using GANs. The GAN-based SIM is compared to the canonical SIM based on its ability to reproduce those statistics that are meaningful to the particular medically realistic SIM considered. It is shown that evaluating GANs using classical metrics and medically relevant metrics may lead to different conclusions about the fidelity of the trained GANs. This work highlights the need for the development of objective metrics for evaluating GANs.

show abstract

Impact of deep learning-based image super-resolution on binary signal detection

Cited by 22 publications

References 48 publications

Prior image-based medical image reconstruction using a style-based generative adversarial network

Prior image-based medical image reconstruction using a style-based generative adversarial network

Estimating task-based performance bounds for image reconstruction methods by use of learned-ideal observers

Evaluating Procedures for Establishing Generative Adversarial Network-based Stochastic Image Models in Medical Imaging

Contact Info

Product

Resources

About