3-D Reconstruction in Canonical Co-Ordinate Space From Arbitrarily Oriented 2-D Images

Hou, Benjamin; Khanal, Bishesh; Alansary, Amir; McDonagh, Steven; Davidson, Alice; Rutherford, Mary A.; Hajnal, Joseph V.; Rueckert, Daniel; Glocker, Ben; Kainz, Bernhard

doi:10.1109/tmi.2018.2798801

Cited by 76 publications

(71 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is demonstrated on the current state-of-the-art for pose estimation, PoseNet, where we show that our method achieves similar performance as the carefully tuned approximation used in [7]. We also show significant improvements for medical image pose estimation and outperform the state-of-the-art in this domain [4,3]. Acknowledgements: Supported by the Wellcome Trust IEH Award [102431] and Nvidia.…”

Section: Resultsmentioning

confidence: 52%

“…We derive appropriate gradients that are required for CNN back propagation. Our method couples the translation and rotation parameters, and regresses them simultaneously as one parameter on the Lie algebra se (3). We show that our loss function is agnostic to the architecture by training different CNNs and can effectively predict poses that are comparable to state-of-the-art methods.…”

Section: Introductionmentioning

confidence: 97%

“…Alternatively, [8,16] reparameterised the pose parameters as projected co-ordinates on a 2D view plane. This was similarly performed by [4,3] with Anchor Points, where three arbitrary selected reference points on a 2D plane define the plane's location in 3D space. Using the L2-norm to calculate the Euclidean distance between a predicted projection co-ordinate and the ground truth projection co-ordinate is appropriate, as the L2-norm is the appropriate metric.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Computing CNN Loss and Gradients for Pose Estimation with Riemannian Geometry

Hou

Miolane²,

Khanal

et al. 2018

Medical Image Computing and Computer Assisted Intervention – MICCAI 2018

Self Cite

View full text Add to dashboard Cite

Pose estimation, i.e. predicting a 3D rigid transformation with respect to a fixed co-ordinate frame in, SE (3), is an omnipresent problem in medical image analysis. Deep learning methods often parameterise poses with a representation that separates rotation and translation. Available frameworks do not provide means to calculate loss on a manifold, regression is usually performed using the L2-norm independently on the rotation's and the translation's parameterisations. This is a metric for linear spaces that does not take into account the Lie group structure of SE(3). We propose a general Riemannian formulation of the pose estimation problem, and train CNNs directly on SE(3) equipped with a left-invariant Riemannian metric. The loss between the ground truth and predicted pose (elements of the manifold) is calculated as the Riemannian geodesic distance, which couples together the translation and rotation components. Network weights are updated by back-propagating the gradient with respect to the predicted pose on the tangent space of the manifold SE(3). We thoroughly evaluate the effectiveness of our loss function by comparing its performance with popular and most commonly used existing methods, on tasks such as image-based localisation and intensity-based 2D/3D registration. We also show that hyper-parameters, used in our loss function to weight the contribution between rotations and translations, can be intrinsically calculated from the dataset to achieve greater performance margins.

show abstract

Section: Resultsmentioning

confidence: 52%

Section: Introductionmentioning

confidence: 97%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Computing CNN Loss and Gradients for Pose Estimation with Riemannian Geometry

Hou

Miolane²,

Khanal

et al. 2018

Medical Image Computing and Computer Assisted Intervention – MICCAI 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…Deep learning has been successfully applied to image registration [29,30], but in nonlinear image registration tasks with high accuracy requirement, such as brain shift correction, further exploration is still needed [31]. The two submissions that used DL in this challenge were from Sun et al [11], who did not participate to the second phase, and Zhong et al [17] (team FAX), who ranked first in the results reported on the training database.…”

Section: Discussionmentioning

confidence: 99%

Evaluation of MRI to Ultrasound Registration Methods for Brain Shift Correction: The CuRIOUS2018 Challenge

Xiao

Maier

Wein

et al. 2020

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

In brain tumor surgery, the quality and safety of the procedure can be impacted by intra-operative tissue deformation, called brain shift. Brain shift can move the surgical targets and other vital structures such as blood vessels, thus invalidating the presurgical plan. Intra-operative ultrasound (iUS) is a convenient and cost-effective imaging tool to track brain shift and tumor resection. Accurate image registration techniques that update pre-surgical MRI based on iUS are crucial but challenging. The MICCAI Challenge 2018 for Correction of Brain shift with Intra-Operative UltraSound (CuRIOUS2018) provided a public platform to benchmark MRI-iUS registration algorithms on newly released clinical datasets. In this work, we present the data, setup, evaluation, and results of CuRIOUS 2018, which received 6 fully automated algorithms from leading academic and industrial research groups. All algorithms were first trained with the public RESECT database, and then ranked based on a test dataset of 10 additional cases with identical data curation and annotation protocols as the RESECT database. The article compares the results of all participating teams and discusses the insights gained from the challenge, as well as future work.

show abstract

“…Prior work on slice-to-volume registration in fetal MRI has shown a strong need for regularization and initialization of slice transformations through hierarchical registration [23], [24] or state-space motion modeling [25]. Learning-based methods have been recently used to improve prediction of slice locations in fetal MRI [26], [27] and fetal ultrasound [28]. In [26], [27] anchor-point slice parametrization was used along with the Euclidean loss function based on [29] to predict slice positions and reconstruct fetal MRI in canonical space.…”

Section: Contributionsmentioning

confidence: 99%

Real-Time Deep Pose Estimation With Geodesic Loss for Image-to-Template Rigid Registration

Salehi

Khan

Erdoğmuş

et al. 2019

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

With an aim to increase the capture range and accelerate the performance of state-of-the-art inter-subject and subject-totemplate 3D rigid registration, we propose deep learning-based methods that are trained to find the 3D position of arbitrarily oriented subjects or anatomy in a canonical space based on slices or volumes of medical images. For this, we propose regression convolutional neural networks (CNNs) that learn to predict the angle-axis representation of 3D rotations and translations using image features. We use and compare mean square error and geodesic loss to train regression CNNs for 3D pose estimation used in two different scenarios: slice-to-volume registration and volume-to-volume registration. As an exemplary application, we applied the proposed methods to register arbitrarily oriented reconstructed images of fetuses scanned in-utero at a wide gestational age range to a standard atlas space. Our results show that in such registration applications that are amendable to learning, the proposed deep learning methods with geodesic loss minimization achieved 3D pose estimation with a wide capture range in real-time (< 100ms). We also tested the generalization capability of the trained CNNs on an expanded age range and on images of newborn subjects with similar and different MR image contrasts. We trained our models on T2-weighted fetal brain MRI scans and used them to predict the 3D pose of newborn brains based on T1-weighted MRI scans. We showed that the trained models generalized well for the new domain when we performed image contrast transfer through a conditional generative adversarial network. This indicates that the domain of application of the trained deep regression CNNs can be further expanded to image modalities and contrasts other than those used in training. A combination of our proposed methods with accelerated optimization-based registration algorithms can dramatically enhance the performance of automatic imaging devices and image processing methods of the future.

show abstract

3-D Reconstruction in Canonical Co-Ordinate Space From Arbitrarily Oriented 2-D Images

Cited by 76 publications

References 42 publications

Computing CNN Loss and Gradients for Pose Estimation with Riemannian Geometry

Computing CNN Loss and Gradients for Pose Estimation with Riemannian Geometry

Evaluation of MRI to Ultrasound Registration Methods for Brain Shift Correction: The CuRIOUS2018 Challenge

Real-Time Deep Pose Estimation With Geodesic Loss for Image-to-Template Rigid Registration

Contact Info

Product

Resources

About