AUE-Net: Automated Generation of Ultrasound Elastography Using Generative Adversarial Network

Abstract

Problem: Ultrasonography is recommended as the first choice for evaluation of thyroid nodules, however, conventional ultrasound features may not be able to adequately predict malignancy. Ultrasound elastography, adjunct to conventional B-mode ultrasound, can effectively improve the diagnostic accuracy of thyroid nodules. However, this technology requires professional elastography equipment and experienced physicians. Aim: in the field of computational medicine, Generative Adversarial Networks (GANs) were proven to be a powerful tool for generating high-quality images. This work therefore utilizes GANs to generate ultrasound elastography images. Methods: this paper proposes a new automated generation method of ultrasound elastography (AUE-net) to generate elastography images from conventional ultrasound images. The AUE-net was based on the U-Net architecture and optimized by attention modules and feature residual blocks, which could improve the adaptability of feature extraction for nodules of different sizes. The additional color loss function was used to balance color distribution. In this network, we first attempted to extract the tissue features of the ultrasound image in the latent space, then converted the attributes by modeling the strain, and finally reconstructed them into the corresponding elastography image. Results: a total of 726 thyroid ultrasound elastography images with corresponding conventional images from 397 patients were obtained between 2019 and 2021 as the dataset (646 in training set and 80 in testing set). The mean rating accuracy of the AUE-net generated elastography images by ultrasound specialists was 84.38%. Compared with that of the existing models in the visual aspect, the presented model generated relatively higher quality elastography images. Conclusion: the AUE-net generated ultrasound elastography images showed natural appearance and retained tissue information. Accordingly, it seems that B-mode ultrasound harbors information that can link to tissue elasticity. This study may pave the way to generate ultrasound elastography images readily without the need for professional equipment.

Unveiling the future of breast cancer assessment: a critical review on generative adversarial networks in elastography ultrasound

“…This thorough analysis of the GAN framework across different medical centers is unique to the study conducted by Yao et al. ( 20 ) and is missing from other studies for elastogram synthesis ( 38 ). However, the validation sets are completely based on the Chinese population, requiring further validation for other ethnic groups.…”

TSE-GAN: strain elastography using generative adversarial network for thyroid disease diagnosis

“…It is important to note that these methods primarily focus on shear wave imaging, which differs from strain imaging in terms of imaging principles. Zhang et al ( Zhang et al, 2022 ) introduced the AUE-Net, which was based on the U-Net architecture and optimized using attention modules and feature residual blocks. However, since their dataset is compression ultrasound images, the raw data acquisition still requires manual compression by operators.…”

“…We choose seven commonly-used image-to-image translation (I2IT) methods. Paired I2IT models include Pix2pix ( Isola et al, 2017 ), Pix2pixHD, LPTN ( Liang et al, 2021 ) and AUE-Net ( Zhang et al, 2022 ), while unpaired I2IT models include CycleGAN ( Zhu et al, 2017 ), AttentionGAN and Qsattn ( Hu et al, 2022 ). We then compare them qualitatively and quantitatively in the Thyroid Strain Elastography dataset.…”

Evaluating Synthetic Medical Images Using Artificial Intelligence with the GAN Algorithm

“…So, in recent years, building synthetic echocardiogram image datasets has received considerable attention [ 3 , 4 , 5 ]. Generative adversarial networks (GAN) [ 6 ], autoencoders (AEs) [ 7 ], and U-nets [ 8 ] have become the most efficient and popular methods to generate synthetic images, and hundreds of their hybrid algorithms have been proposed so far [ 9 , 10 ].…”