Evaluating procedures for establishing generative adversarial network-based stochastic image models in medical imaging

Kelkar, Varun A.; Gotsis, Dimitrios S.; Brooks, Frank J.; Myers, Kyle J.; Kc, Prabhat; Zeng, Rongping; Anastasio, Mark A.

doi:10.1117/12.2612893

“…Moreover, the concern that standard reference images for multimodal fusion images are not available in the clinical setting will be mitigated. A series of GAN-based work has also been carried out by researchers related to image quality evaluation (Ma et al, 2019;Guo et al, 2023;Kelkar et al, 2023;Li and He, 2024). In 2019, Ma et al (2019) proposed an end-to-end GAN model for quality assessment of images based on multitasking.…”

Section: Gan-based Image Quality Assessmentmentioning

confidence: 99%

G2NPAN: GAN-guided nuance perceptual attention network for multimodal medical fusion image quality assessment

Tian,

Zhang

2024

Front. Neurosci.

0

View full text Add to dashboard Cite

Multimodal medical fusion images (MMFI) are formed by fusing medical images of two or more modalities with the aim of displaying as much valuable information as possible in a single image. However, due to the different strategies of various fusion algorithms, the quality of the generated fused images is uneven. Thus, an effective blind image quality assessment (BIQA) method is urgently required. The challenge of MMFI quality assessment is to enable the network to perceive the nuances between fused images of different qualities, and the key point for the success of BIQA is the availability of valid reference information. To this end, this work proposes a generative adversarial network (GAN) -guided nuance perceptual attention network (G2NPAN) to implement BIQA for MMFI. Specifically, we achieve the blind evaluation style via the design of a GAN and develop a Unique Feature Warehouse module to learn the effective features of fused images from the pixel level. The redesigned loss function guides the network to perceive the image quality. In the end, the class activation mapping supervised quality assessment network is employed to obtain the MMFI quality score. Extensive experiments and validation have been conducted in a database of medical fusion images, and the proposed method is superior to the state-of-the-art BIQA method.

show abstract

“…Summary measures computed from these statistical quantities were compared against the FID for the purpose of assessing the fidelity of the trained GAN. This work is an extension of a preliminary study conducted using an angiographic SIM [33].…”

Section: Introductionmentioning

confidence: 95%

Assessing the Ability of Generative Adversarial Networks to Learn Canonical Medical Image Statistics

Kelkar

¹

,

Gotsis

²

,

Brooks

³

et al. 2023

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for potential applications in medical imaging, such as medical image synthesis, restoration, reconstruction, translation, as well as objective image quality assessment. Despite the impressive progress in generating high-resolution, perceptually realistic images, it is not clear if modern GANs reliably learn the statistics that are meaningful to a downstream medical imaging application. In this work, the ability of a state-of-the-art GAN to learn the statistics of canonical stochastic image models (SIMs) that are relevant to objective assessment of image quality is investigated. It is shown that although the employed GAN successfully learned several basic first-and second-order statistics of the specific medical SIMs under consideration and generated images with high perceptual quality, it failed to correctly learn several per-image statistics pertinent to the these SIMs, highlighting the urgent need to assess medical image GANs in terms of objective measures of image quality.

show abstract

“…Summary measures computed from these identified statistical quantities were compared against the FID for the purpose of assessing the fidelity of the trained GAN. This work is an extension of a preliminary study conducted using an angiographic SIM [33].…”

Section: Introductionmentioning

confidence: 95%

Assessing the ability of generative adversarial networks to learn canonical medical image statistics

Kelkar¹,

Gotsis²,

Brooks³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for potential applications in medical imaging, such as medical image synthesis, restoration, reconstruction, translation, as well as objective image quality assessment. Despite the impressive progress in generating high-resolution, perceptually realistic images, it is not clear if modern GANs reliably learn the statistics that are meaningful to a downstream medical imaging application. In this work, the ability of a state-of-the-art GAN to learn the statistics of canonical stochastic image models (SIMs) that are relevant to objective assessment of image quality is investigated. It is shown that although the employed GAN successfully learned several basic first-and second-order statistics of the specific medical SIMs under consideration and generated images with high perceptual quality, it failed to correctly learn several per-image statistics pertinent to the these SIMs, highlighting the urgent need to assess medical image GANs in terms of objective measures of image quality.

show abstract

Evaluating procedures for establishing generative adversarial network-based stochastic image models in medical imaging

Cited by 3 publications

References 12 publications

G2NPAN: GAN-guided nuance perceptual attention network for multimodal medical fusion image quality assessment

G2NPAN: GAN-guided nuance perceptual attention network for multimodal medical fusion image quality assessment

Assessing the Ability of Generative Adversarial Networks to Learn Canonical Medical Image Statistics

Assessing the ability of generative adversarial networks to learn canonical medical image statistics

Contact Info

Product

Resources

About