Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation

Abstract

Purpose: We present an original method for simulating realistic fetal neurosonography images specifically generating third-trimester pregnancy ultrasound images from second-trimester images. Our method was developed using unpaired data as pairwise data were not available. We also report original insights on the general appearance differences between second-and third-trimester fetal head transventricular (TV) plane images.Approach: We design a cycle-consistent adversarial network (Cycle-GAN) to simulate visually realistic third-trimester images from unpaired second-and third-trimester ultrasound images. Simulation realism is evaluated qualitatively by experienced sonographers who blindly graded real and simulated images. A quantitative evaluation is also performed whereby a validated deeplearning-based image recognition algorithm (ScanNav ® ) acts as the expert reference to allow hundreds of real and simulated images to be automatically analyzed and compared efficiently.Results: Qualitative evaluation shows that the human expert cannot tell the difference between real and simulated third-trimester scan images. 84.2% of the simulated third-trimester images could not be distinguished from the real third-trimester images. As a quantitative baseline, on 3000 images, the visibility drop of the choroid, CSP, and mid-line falx between real second-and real third-trimester scans was computed by ScanNav ® and found to be 72.5%, 61.5%, and 67%, respectively. The visibility drop of the same structures between real second-trimester and simulated third-trimester was found to be 77.5%, 57.7%, and 56.2%, respectively. Therefore, the real and simulated third-trimester images were consider to be visually similar to each other. Our evaluation also shows that the third-trimester simulation of a conventional GAN is much easier to distinguish, and the visibility drop of the structures is smaller than our proposed method. Conclusions:The results confirm that it is possible to simulate realistic third-trimester images from second-trimester images using a modified Cycle-GAN, which may be useful for deep learning researchers with a restricted availability of third-trimester scans but with access to ample second trimester images. We also show convincing simulation improvements, both qualitatively and quantitatively, using the Cycle-GAN method compared with a conventional GAN. Finally, the use of a machine learning-based reference (in the case ScanNav ® ) for large-scale quantitative image analysis evaluation is also a first to our knowledge.

Development and validation of an artificial intelligence assisted prenatal ultrasonography screening system for trainees

“…Most AI models developed for prenatal diagnosis have a limited gestational age range for which they are applicable (usually 18–22 weeks), as the availability of third‐trimester data is limited 33 . The PSAIS developed in this study can be applied to a much wider range of gestational ages because of the gestational characteristics of our training dataset.…”

Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology

“…In particular, the method by Di Vece et al used a 23-week synthetic fetal phantom for system development and was the only study to estimate the 6D poses of US planes combining common 3D planes with rotation around the brain center [82]. Xu et al presented an AI method for authentically simulating third-trimester images from second-trimester images for deep-learning researchers with restricted access to third-trimester images [84]. The automated detection of brain structures and malformations was described by Lin et al [85,86], Alansary et al [87], and Gofer et al [88] in 2D images and videos, and in 3D volumes by Hesse et al and Huang et al [79,89].…”

“…The primary limitation of AI US imaging in this topic was described to be the rapid anatomical development of fetal brain structures due to brain maturation, increasing head size and degree of ossification with rising GA [78,80,84]. Ossification of the fetal skull provoked an increase in the shadowing of US images and thus reduced image quality and visibility [80].…”

Unleashing the AI revolution: exploring the capabilities and challenges of large language models and text‐to‐image AI programs

“…Recently, the ability to generate realistic third-trimester images of the transventricular plane of the fetal head through simulation using second-trimester images has been demonstrated 12 . However, so far, AI programs like Midjourney appear not to be able to create reliable medical material, due to intentional restrictions by the developers.…”