Weakly-Supervised Biomechanically-Constrained CT/MRI Registration of the Spine

Jian, Bailiang; Azampour, Mohammad Farid; Benetti, Francesca; Oberreuter, Johannes; Bukas, Christina; Gersing, Alexandra S.; Foreman, Sarah C.; Dietrich, Anna-Sophia; Rischewski, Jon; Kirschke, Jan S.; Navab, Nassir; Wendler, Thomas

doi:10.1007/978-3-031-16446-0_22

Cited by 6 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study introduces a novel deep learning pipeline for CT‐to‐US spine registration, incorporating anatomical losses to facilitate learning of spinal biomechanics. Although biomechanical constraints were employed for DL approaches trained for other modalities (MRI‐CT) 21 or other organs (prostate), 18 application to CT‐to‐US spine registration was not investigated. Additionally, we propose a data generation method to overcome the challenge of procuring sufficient paired CT‐US data, marking a novel approach in this domain.…”

Section: Discussionmentioning

confidence: 99%

“…However, it has been shown in non DL methods that regularizing the registration via the introduction of biomechanical constraints yields improve performance. 7,13 DL approaches also show improvement in the performance and feasibility of the inferred deformation field when biomechanical constraints are added to the training phase, for example, for magnetic resonance imaging (MRI) to CT images registration 21 or registration of images of the prostate. 18 In this paper, we propose a deep-learning-based pipeline for CT-to-US spine registration and a novel fully automatic data generation method, to effectively generate paired CT-US images for network training.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Anatomy‐aware computed tomography‐to‐ultrasound spine registration

Azampour,

Tirindelli,

Lameski

et al. 2023

Medical Physics

Self Cite

View full text Add to dashboard Cite

BackgroundUltrasound (US) has demonstrated to be an effective guidance technique for lumbar spine injections, enabling precise needle placement without exposing the surgeon or the patient to ionizing radiation. However, noise and acoustic shadowing artifacts make US data interpretation challenging. To mitigate these problems, many authors suggested using computed tomography (CT)‐to‐US registration to align the spine in pre‐operative CT to intra‐operative US data, thus providing localization of spinal landmarks.PurposeIn this paper, we propose a deep learning (DL) pipeline for CT‐to‐US registration and address the problem of a need for annotated medical data for network training. Firstly, we design a data generation method to generate paired CT‐US data where the spine is deformed in a physically consistent manner. Secondly, we train a point cloud (PC) registration network using anatomy‐aware losses to enforce anatomically consistent predictions.MethodsOur proposed pipeline relies on training the network on realistic generated data. In our data generation method, we model the properties of the joints and disks between vertebrae based on biomechanical measurements in previous studies. We simulate the supine and prone position deformation by applying forces on the spine models. We choose the spine models from 35 patients in VerSe dataset. Each spine is deformed 10 times to create a noise‐free data with ground‐truth segmentation at hand. In our experiments, we use one‐leave‐out cross‐validation strategy to measure the performance and the stability of the proposed method. For each experiment, we choose generated PCs from three spines as the test set. From the remaining, data from 3 spines act as the validation set and we use the rest of the data for training the algorithm.To train our network, we introduce anatomy‐aware losses and constraints on the movement to match the physics of the spine, namely, rigidity loss and bio‐mechanical loss. We define rigidity loss based on the fact that each vertebra can only transform rigidly while the disks and the surrounding tissue are deformable. Second, by using bio‐mechanical loss we stop the network from inferring extreme movements by penalizing the force needed to get to a certain pose.ResultsTo validate the effectiveness of our fully automated data generation pipeline, we qualitatively assess the fidelity of the generated data. This assessment involves verifying the realism of the spinal deformation and subsequently confirming the plausibility of the simulated ultrasound images. Next, we demonstrate that the introduction of the anatomy‐aware losses brings us closer to state‐of‐the‐art (SOTA) and yields a reduction of 0.25 mm in terms of target registration error (TRE) compared to using only mean squared error (MSE) loss on the generated dataset. Furthermore, by using the proposed losses, the rigidity loss in inference decreases which shows that the inferred deformation respects the rigidity of the vertebrae and only introduces deformations in the soft tissue area to compensate the difference to the target PC. We also show that our results are close to the SOTA for the simulated US dataset with TRE of 3.89 mm and 3.63 mm for the proposed method and SOTA respectively. In addition, we show that our method is more robust against errors in the initialization in comparison to SOTA and significantly achieves better results (TRE of 4.88 mm compared to 5.66 mm) in this experiment.ConclusionsIn conclusion, we present a pipeline for spine CT‐to‐US registration and explore the potential benefits of utilizing anatomy‐aware losses to enhance registration results. Additionally, we propose a fully automatic method to synthesize paired CT‐US data with physically consistent deformations, which offers the opportunity to generate extensive datasets for network training.The generated dataset and the source code for data generation and registration pipeline can be accessed via https://github.com/mfazampour/medphys_ct_us_registration.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Anatomy‐aware computed tomography‐to‐ultrasound spine registration

Azampour,

Tirindelli,

Lameski

et al. 2023

Medical Physics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Reviews (Boveiri et al 2020) neural networks (CNNs) (Cao et al 2018, Ferrante et al 2018, Hu et al 2018, Balakrishnan et al 2018, Kim et al 2019, Kuang and Schmah 2019, Jian et al 2022, Wolterink et al 2022, Xi et al 2022, Liang et al 2023. CNNs have been researched for direct DVF regression, with reported improvements in DVF when coupled with spatial transformer networks (Jaderberg et al 2015).…”

Section: Direct Determination Of Dvfs By Machine Learning Algorithmsmentioning

confidence: 99%

“…Reviews (Boveiri et al 2020 ) cover a range of algorithm architectures. DL architectures include staked auto-encoders (SAEs) (Wang et al 2017 , Krebs et al 2018 ), bayesian frameworks (Deshpande and Bhatt 2019 , Khawaled and Freiman 2020 , 2022a ), implicit neural representations (Wolterink et al 2022 ) and convolution neural networks (CNNs) (Cao et al 2018 , Ferrante et al 2018 , Hu et al 2018 , Balakrishnan et al 2018 , 2019 , Kim et al 2019 , Kuang and Schmah 2019 , Liu et al 2019 , Jian et al 2022 , Wolterink et al 2022 , Xi et al 2022 , Liang et al 2023 ). CNNs have been researched for direct DVF regression, with reported improvements in DVF when coupled with spatial transformer networks (Jaderberg et al 2015 ).…”

Section: Dir Algorithmsmentioning

confidence: 99%

Review and recommendations on deformable image registration uncertainties for radiotherapy applications

Nenoff,

Amstutz,

Murr

et al. 2023

Phys. Med. Biol.

View full text Add to dashboard Cite

Deformable image registration (DIR) is a versatile tool used in many applications in radiotherapy (RT). DIR algorithms have been implemented in many commercial treatment planning systems providing accessible and easy-to-use solutions. However, the geometric uncertainty of DIR can be large and difficult to quantify, resulting in barriers to clinical practice. Currently, there is no agreement in the RT community on how to quantify these uncertainties and determine thresholds that distinguish a good DIR result from a poor one. This review summarises the current literature on sources of DIR uncertainties and their impact on RT applications. Recommendations are provided on how to handle these uncertainties for patient-specific use, commissioning, and research. Recommendations are also provided for developers and vendors to help users to understand DIR uncertainties and make the application of DIR in RT safer and more reliable.

show abstract