Deep similarity learning using a siamese ResNet trained on similarity labels from disparity maps of cerebral MRA MIP pairs

Neumann, Christian; Tönnies, Klaus D.; Pohle-Fröhlich, Regina

doi:10.1117/12.2548576

Cited by 4 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is well known that most commonly used metrics fail to accurately represent the distances in geometric parameter space that generate the mismatch between the current observations. It is thus not surprising that ten studies describe methods to better model and quantify the similarity between the source and target images to increase the capture range, and thus, the likelihood of registration success (Tang and Scalzo, 2016;Liao et al, 2019;Schaffert et al, 2019Schaffert et al, , 2020bSchaffert et al, 2020a;Gao et al, 2020c;Grupp et al, 2020c;Francois et al, 2020;Gu et al, 2020;Neumann et al, 2020). Some studies propose novel image similarity functions S θS (•, •) that, analogous to traditional similarity metrics, accept as input the source and target image and return a scalar or vector that is related to the mismatch in parameter space (Francois et al, 2020;Gu et al, 2020;Tang and Scalzo, 2016;Neumann et al, 2020;Grupp et al, 2020c;Gao et al, 2020b).…”

Section: Similarity Modelingmentioning

confidence: 99%

“…It is thus not surprising that ten studies describe methods to better model and quantify the similarity between the source and target images to increase the capture range, and thus, the likelihood of registration success (Tang and Scalzo, 2016;Liao et al, 2019;Schaffert et al, 2019Schaffert et al, , 2020bSchaffert et al, 2020a;Gao et al, 2020c;Grupp et al, 2020c;Francois et al, 2020;Gu et al, 2020;Neumann et al, 2020). Some studies propose novel image similarity functions S θS (•, •) that, analogous to traditional similarity metrics, accept as input the source and target image and return a scalar or vector that is related to the mismatch in parameter space (Francois et al, 2020;Gu et al, 2020;Tang and Scalzo, 2016;Neumann et al, 2020;Grupp et al, 2020c;Gao et al, 2020b). Among those, two methods rely on regularization: Grupp et al (2020c) detect anatomical landmarks to expand an analytic similarity function with landmark-reprojection constraints to enhance the capture range of an intensity-based strategy, while Francois et al (2020) segment occluding contours to constrain similarity evaluation to salient regions.…”

Section: Similarity Modelingmentioning

confidence: 99%

Section: Systematic Reviewmentioning

confidence: 99%

“…Some studies propose novel image similarity functions

that, analogous to traditional similarity metrics, accept as input the source and target image and return a scalar or vector that is related to the mismatch in parameter space ( Francois et al, 2020 ; Gu et al, 2020 ; Tang and Scalzo, 2016 ; Neumann et al, 2020 ; Grupp et al, 2020c ; Gao et al, 2020b ). Among those, two methods rely on regularization: Grupp et al (2020c) detect anatomical landmarks to expand an analytic similarity function with landmark-reprojection constraints to enhance the capture range of an intensity-based strategy, while Francois et al (2020) segment occluding contours to constrain similarity evaluation to salient regions.…”

Section: Systematic Reviewmentioning

confidence: 99%

“…To this end, Gu et al (2020) and Gao et al (2020b) estimate the geodesic in Riemannian tangent space between the source and target camera poses, which in an ideal case results in a convex similarity function. Tang and Scalzo (2016) learn a more expressive feature descriptor to better quantify the mismatch in vasculature registration, and for a similar application, Neumann et al (2020) regress the disparity between corresponding points in source and target images to quantify image dissimilarity. Both methods rely on concepts that are limited to sparse objects, such as vessels.…”

Section: Systematic Reviewmentioning

confidence: 99%

See 4 more Smart Citations

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective

Unberath

Gao

et al. 2021

Front. Robot. AI

View full text Add to dashboard Cite

Image-based navigation is widely considered the next frontier of minimally invasive surgery. It is believed that image-based navigation will increase the access to reproducible, safe, and high-precision surgery as it may then be performed at acceptable costs and effort. This is because image-based techniques avoid the need of specialized equipment and seamlessly integrate with contemporary workflows. Furthermore, it is expected that image-based navigation techniques will play a major role in enabling mixed reality environments, as well as autonomous and robot-assisted workflows. A critical component of image guidance is 2D/3D registration, a technique to estimate the spatial relationships between 3D structures, e.g., preoperative volumetric imagery or models of surgical instruments, and 2D images thereof, such as intraoperative X-ray fluoroscopy or endoscopy. While image-based 2D/3D registration is a mature technique, its transition from the bench to the bedside has been restrained by well-known challenges, including brittleness with respect to optimization objective, hyperparameter selection, and initialization, difficulties in dealing with inconsistencies or multiple objects, and limited single-view performance. One reason these challenges persist today is that analytical solutions are likely inadequate considering the complexity, variability, and high-dimensionality of generic 2D/3D registration problems. The recent advent of machine learning-based approaches to imaging problems that, rather than specifying the desired functional mapping, approximate it using highly expressive parametric models holds promise for solving some of the notorious challenges in 2D/3D registration. In this manuscript, we review the impact of machine learning on 2D/3D registration to systematically summarize the recent advances made by introduction of this novel technology. Grounded in these insights, we then offer our perspective on the most pressing needs, significant open problems, and possible next steps.

show abstract

Section: Similarity Modelingmentioning

confidence: 99%

Section: Similarity Modelingmentioning

confidence: 99%

Section: Systematic Reviewmentioning

confidence: 99%

“…Some studies propose novel image similarity functions

Section: Systematic Reviewmentioning

confidence: 99%

Section: Systematic Reviewmentioning

confidence: 99%

See 3 more Smart Citations

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective

Unberath

Gao

et al. 2021

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

SGReg: segmentation guided 3D/2D rigid registration for orthogonal X-ray and CT images in spine surgery navigation

Wang

Zhao

et al. 2023

Phys. Med. Biol.

View full text Add to dashboard Cite

Objective. One of the essential technologies in various image-guided spine surgeries is the rigid registration of 3D pre- CT and 2D intra-operative X-ray images. The 3D/2D registration is patterned as two essential tasks, that is, dimensional correspondence establishment and estimation of the 3D pose. 3D data is projected to 2D for dimensional correspondence by most of the existing methods, which makes pose parameters difficult to estimate caused by the loss of spatial information. This work aims to develop a reconstruction based 3D/2D registration method for spine surgery navigation. Approach. A novel segmentation-guided 3D/2D registration (SGReg) method for orthogonal X-ray and CT images was proposed based on reconstruction. SGReg consists of a bi-path segmentation network and an inter-path multi-scale pose estimation module. The X-ray segmentation path in the bi-path segmentation network reconstructs 3D spatial information from 2D orthogonal X-ray images to segmentation maps; meanwhile, the CT segmentation path predicts segmentation maps from 3D CT images, thereby bringing the 3D/2D data into dimensional correspondence. In the inter-path multi-scale pose estimation module, the features from the two segmentation paths are integrated, and the pose parameters are directly regressed under the guidance of the coordinate information. Main Results. We evaluated SGReg using a public dataset CTSpine1k and compared the registration performance with other learning-based methods. SGReg achieved considerable improvement over other methods with great robustness . Significance. We have proposed an end-to-end 3D/2D registration framework named SGReg. Based on the idea of reconstruction, SGReg performs a unified framework between dimensional correspondence establishment and direct pose estimation in 3D space, showing significant potential in spine surgery navigation.

show abstract

X-Ray to DRR Images Translation for Efficient Multiple Objects Similarity Measures in Deformable Model 3D/2D Registration

Aubert

Cresson

Guise

et al. 2023

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Deep similarity learning using a siamese ResNet trained on similarity labels from disparity maps of cerebral MRA MIP pairs

Cited by 4 publications

References 0 publications

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective

SGReg: segmentation guided 3D/2D rigid registration for orthogonal X-ray and CT images in spine surgery navigation

X-Ray to DRR Images Translation for Efficient Multiple Objects Similarity Measures in Deformable Model 3D/2D Registration

Contact Info

Product

Resources

About