Deep learning-based plane pose regression in obstetric ultrasound

Abstract

Purpose In obstetric ultrasound (US) scanning, the learner’s ability to mentally build a three-dimensional (3D) map of the fetus from a two-dimensional (2D) US image represents a major challenge in skill acquisition. We aim to build a US plane localisation system for 3D visualisation, training, and guidance without integrating additional sensors. Methods We propose a regression convolutional neural network (CNN) using image features to estimate the six-dimensional pose of arbitrarily oriented US planes relative to the fetal brain centre. The network was trained on synthetic images acquired from phantom 3D US volumes and fine-tuned on real scans. Training data was generated by slicing US volumes into imaging planes in Unity at random coordinates and more densely around the standard transventricular (TV) plane. Results With phantom data, the median errors are 0.90 mm/1.17$$^\circ $$ ∘ and 0.44 mm/1.21$$^\circ $$ ∘ for random planes and planes close to the TV one, respectively. With real data, using a different fetus with the same gestational age (GA), these errors are 11.84 mm/25.17$$^\circ $$ ∘ . The average inference time is 2.97 ms per plane. Conclusion The proposed network reliably localises US planes within the fetal brain in phantom data and successfully generalises pose regression for an unseen fetal brain from a similar GA as in training. Future development will expand the prediction to volumes of the whole fetus and assess its potential for vision-based, freehand US-assisted navigation when acquiring standard fetal planes.

Artificial intelligence in obstetric ultrasound: A scoping review

“…84 Twenty studies focused on the use of AI for fetal neurosonography beyond the first trimester (Table 5). Three studies evaluated deep learning algorithms to localize planes within the fetal brain from 3D ultrasound volumes, [85][86][87] whereas four additional studies proposed models to segment or measure various intracranial structures from 3D US volumes of the fetal head. [88][89][90][91] Two studies focused on deep learning systems for the detection of fetal intracranial planes from 2D ultrasound.…”

Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology

“…The research topics in this field are heterogenous, presenting a wide variety of applications for AI-assisted methods. The establishment of a plane localization system as a 3D reference space for locating 2D planes was proposed by Yeung et [78,80,82,83]. In particular, the method by Di Vece et al used a 23-week synthetic fetal phantom for system development and was the only study to estimate the 6D poses of US planes combining common 3D planes with rotation around the brain center [82].…”

“…The establishment of a plane localization system as a 3D reference space for locating 2D planes was proposed by Yeung et [78,80,82,83]. In particular, the method by Di Vece et al used a 23-week synthetic fetal phantom for system development and was the only study to estimate the 6D poses of US planes combining common 3D planes with rotation around the brain center [82]. Xu et al presented an AI method for authentically simulating third-trimester images from second-trimester images for deep-learning researchers with restricted access to third-trimester images [84].…”

“…Ossification of the fetal skull provoked an increase in the shadowing of US images and thus reduced image quality and visibility [80]. To address the heterogeneity in brain images from different GA, studies described the need for matching GA of US images in algorithms [82,94]. Other study limitations were the missing training of AI algorithms with images of pathologies [79,86,95] and the problem of miscalculations when US images were not in accordance with the guidelines for standard planes [6,80].…”

Gaze-probe joint guidance with multi-task learning in obstetric ultrasound scanning

“…In obstetric scanning guidance, a common practice is to treat probe guidance as an image-guided navigation problem. For example, Di Vece et al (2022) regressed six-dimensional pose of the US head plane relative to the center of fetal brain using phantom data. Zhao et al (2021) proposed to position the probe based on landmark-based image retrieval.…”