Probe localization from ultrasound image sequences using deep learning for volume reconstruction

Miura, Kanta; Ito, Koichi; Aoki, Takafumi; Ohmiya, Jun; Kondo, Satoshi

doi:10.1117/12.2590805

Cited by 7 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Importantly, the consistency loss is a form of "teacher forcing" commonly adopted in training sequence models, which makes use of the ground-truth targets, rather than the previous predictions, to supervise the subsequent prediction during training. It has been proven advantageous in sequence-to-sequence models [14].…”

Section: Loss Functionsmentioning

confidence: 99%

See 1 more Smart Citation

Trackerless freehand ultrasound with sequence modelling and auxiliary transformation over past and future frames

Li¹,

Shen²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Three-dimensional (3D) freehand ultrasound (US) reconstruction without a tracker can be advantageous over its two-dimensional or tracked counterparts in many clinical applications. In this paper, we propose to estimate 3D spatial transformation between US frames from both past and future 2D images, using feed-forward and recurrent neural networks (RNNs). With the temporally available frames, a further multitask learning algorithm is proposed to utilise a large number of auxiliary transformation-predicting tasks between them. Using more than 40,000 US frames acquired from 228 scans on 38 forearms of 19 volunteers in a volunteer study, the hold-out test performance is quantified by frame prediction accuracy, volume reconstruction overlap, accumulated tracking error and final drift, based on ground-truth from an optical tracker. The results show the importance of modelling the temporal-spatially correlated input frames as well as output transformations, with further improvement owing to additional past and/or future frames. The best performing model was associated with predicting transformation between moderately-spaced frames, with an interval of less than ten frames at 20 frames per second (fps). Little benefit was observed by adding frames more than one second away from the predicted transformation, with or without LSTM-based RNNs. Interestingly, with the proposed approach, explicit within-sequence loss that encourages consistency in composing transformations or minimises accumulated error may no longer be required. The implementation code and volunteer data 1 will be made publicly available ensuring reproducibility and further research.

show abstract

Section: Loss Functionsmentioning

confidence: 99%

“…In [12], ResNet and FlowNetS were integrated for a better localization and optical flow estimation, and consistency loss derived from stereo vision was added. Forward consistency loss was then proposed in [14]. In [13], RNN was used to estimate both relative and absolute probe poses.…”

Section: Introductionmentioning

confidence: 99%

Trackerless freehand ultrasound with sequence modelling and auxiliary transformation over past and future frames

Li¹,

Shen²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To enable a smooth 3D reconstruction, a case-wise correlation loss based on 3D CNN and Pearson correlation coefficient was proposed in [10,12]. Qi et al [13] leverages past and future frames to estimate the relative transformation between each pair of the sequence; they used the consistency loss proposed in [14]. Despite the success of these approaches, they still suffer significant cumulative drift errors and mainly focus on linear probe motions.…”

Section: Introductionmentioning

confidence: 99%

Trackerless Volume Reconstruction from Intraoperative Ultrasound Images

El hadramy,

Verde,

Beaudet

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper proposes a method for trackerless ultrasound volume reconstruction in the context of minimally invasive surgery. It is based on a Siamese architecture, including a recurrent neural network that leverages the ultrasound image features and the optical flow to estimate the relative position of frames. Our method does not use any additional sensor and was evaluated on ex vivo porcine data. It achieves translation and orientation errors of 0.449 ± 0.189 mm and 1.3 ± 1.5 degrees respectively for the relative pose estimation. In addition, despite the predominant non-linearity motion in our context, our method achieves a good reconstruction with final and average drift rates of 23.11% and 28.71% respectively. To the best of our knowledge, this is the first work to address volume reconstruction in the context of intravascular ultrasound. Source code of this work is publicly available at https://github.com/Sidaty1/IVUS Trakerless Volume Reconstruction.

show abstract

Pose Estimation of 2D Ultrasound Probe from Ultrasound Image Sequences Using CNN and RNN

Miura

Ito

Aoki

et al. 2021

Simplifying Medical Ultrasound

View full text Add to dashboard Cite

Probe localization from ultrasound image sequences using deep learning for volume reconstruction

Cited by 7 publications

References 5 publications

Trackerless freehand ultrasound with sequence modelling and auxiliary transformation over past and future frames

Trackerless freehand ultrasound with sequence modelling and auxiliary transformation over past and future frames

Trackerless Volume Reconstruction from Intraoperative Ultrasound Images

Pose Estimation of 2D Ultrasound Probe from Ultrasound Image Sequences Using CNN and RNN

Contact Info

Product

Resources

About