Deep Learning for Sensorless 3D Freehand Ultrasound Imaging

Prevost, Raphaël; Salehi, Mehrdad; Sprung, Julian; Ladikos, Alexander; Bauer, Robert S.; Wein, Wolfgang

doi:10.1007/978-3-319-66185-8_71

Cited by 31 publications

(15 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• First, the 3D location of each 2D ultrasound image is estimated, where external sensor tracking is required at either the training [15,16] or inference [4,5] stage, subject to errors caused by subjects' internal motion (e.g. fetal movement).…”

Section: Conventional 3d Reconstruction Approachesmentioning

confidence: 99%

ImplicitVol: Sensorless 3D Ultrasound Reconstruction with Deep Implicit Representation

Yeung¹,

Hesse²,

Aliasi³

et al. 2021

Preprint

View full text Add to dashboard Cite

The objective of this work is to achieve sensorless reconstruction of a 3D volume from a set of 2D freehand ultrasound images with deep implicit representation. In contrast to the conventional way that represents a 3D volume as a discrete voxel grid, we do so by parameterizing it as the zero level-set of a continuous function, i.e. implicitly representing the 3D volume as a mapping from the spatial coordinates to the corresponding intensity values. Our proposed model, termed as ImplicitVol, takes a set of 2D scans and their estimated locations in 3D as input, jointly refining the estimated 3D locations and learning a full reconstruction of the 3D volume. When testing on real 2D ultrasound images, novel cross-sectional views that are sampled from ImplicitVol show significantly better visual quality than those sampled from existing reconstruction approaches, outperforming them by over 30% (NCC and SSIM), between the output and ground-truth on the 3D volume testing data. The code will be made publicly available.

show abstract

Section: Conventional 3d Reconstruction Approachesmentioning

confidence: 99%

ImplicitVol: Sensorless 3D Ultrasound Reconstruction with Deep Implicit Representation

Yeung¹,

Hesse²,

Aliasi³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…While trackerless approaches have not yet been explored in OI, such approaches were first explored more than 10 years ago in ultrasound [12], ,and subsequently improved by using speckle tracking or RF signal-correlation [1]. Recently, a learning-based method for sensorless 3D-US imaging was employed in a deep-learning framework for pose estimation from a series of US images [13]. From the same group, a registration-based panoramic compounding from 3D to panoramic data was presented, employing the well-known LC2 registration similarity metric for image alignment [11].…”

Section: Related Workmentioning

confidence: 99%

Trackerless panoramic optoacoustic imaging: a first feasibility evaluation

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Most previous researches are based on the non-deep learning methods [10,12]. Prevost et al [15] pioneered the deep learning based estimation of relative motion between US images, and later they extended their works by introducing the extra optical flow [2] and integrating a sensor source [14]. Guo et al [3] proposed a deep contextual learning network for reconstruction, which utilizes 3D convolutions over US video clips for feature extraction.…”

Section: Introductionmentioning

confidence: 99%

Self Context and Shape Prior for Sensorless Freehand 3D Ultrasound Reconstruction

Luo

Yang

Huang

et al. 2021

Preprint

View full text Add to dashboard Cite

3D ultrasound (US) is widely used for its rich diagnostic information. However, it is criticized for its limited field of view. 3D freehand US reconstruction is promising in addressing the problem by providing broad range and freeform scan. The existing deep learning based methods only focus on the basic cases of skill sequences, and the model relies on the training data heavily. The sequences in real clinical practice are a mix of diverse skills and have complex scanning paths. Besides, deep models should adapt themselves to the testing cases with prior knowledge for better robustness, rather than only fit to the training cases. In this paper, we propose a novel approach to sensorless freehand 3D US reconstruction considering the complex skill sequences. Our contribution is three-fold. First, we advance a novel online learning framework by designing a differentiable reconstruction algorithm. It realizes an end-to-end optimization from section sequences to the reconstructed volume. Second, a self-supervised learning method is developed to explore the context information that reconstructed by the testing data itself, promoting the perception of the model. Third, inspired by the effectiveness of shape prior, we also introduce adversarial training to strengthen the learning of anatomical shape prior in the reconstructed volume. By mining the context and structural cues of the testing data, our online learning methods can drive the model to handle complex skill sequences. Experimental results on developmental dysplasia of the hip US and fetal US datasets show that, our proposed method can outperform the start-of-the-art methods regarding the shift errors and path similarities.

show abstract

Deep Learning for Sensorless 3D Freehand Ultrasound Imaging

Cited by 31 publications

References 10 publications

ImplicitVol: Sensorless 3D Ultrasound Reconstruction with Deep Implicit Representation

ImplicitVol: Sensorless 3D Ultrasound Reconstruction with Deep Implicit Representation

Trackerless panoramic optoacoustic imaging: a first feasibility evaluation

Self Context and Shape Prior for Sensorless Freehand 3D Ultrasound Reconstruction

Contact Info

Product

Resources

About