Real-time multimodal image registration with partial intraoperative point-set data

Abstract

Highlights Network predicts non-rigid point displacements for of MR-TRUS prostate volume registration. Adopts “model-free” deformation via data-driven learning without heuristic constraints. Network architecture accepts variable number of points in training or at inference. Registration accuracy on sparse data similar to complete data in MR-TRUS registration.

A review of artificial intelligence in prostate cancer detection on imaging

“…Registration of preoperative MRI and intra-operative ultrasound is necessary for guiding MRI-ultrasound fusion biopsy, 156,157 focal therapy, 158,159 and radiotherapy planning on MRI. 160 However, registration of the two different imaging modalities, MRI and ultrasound, is complicated due to (a) the difference in the underlying MR and ultrasound imaging processes, and (b) the deformation between the two imaging procedures.…”

“…In an attempt to improve registration between the two modalities, several studies used pre-defined corresponding anatomical structures. [156][157][158][159]161,162 Some approaches used deformable transformations 161 to model patient movement, surrounding organs, for example, bladder and rectum, or interaction with surgical instrument, for example, biopsy needles and ultrasound probes. Others used AI models without constrained transformation models, [156][157][158] or prior knowledge in modeling soft tissue motion.…”

“…[156][157][158][159]161,162 Some approaches used deformable transformations 161 to model patient movement, surrounding organs, for example, bladder and rectum, or interaction with surgical instrument, for example, biopsy needles and ultrasound probes. Others used AI models without constrained transformation models, [156][157][158] or prior knowledge in modeling soft tissue motion. 163,164 AI models have also been proposed to learn similarity measures 165 or transformation models from either biomechanical simulations (which emphasize biologically meaningful registration) 159 or shape populations.…”

“…AI models processing point set data 167 for registration often represent corresponding structures without detailed voxel-level correspondence. 156,157,164 An advantage of using point set is its robustness to scanning protocols, largely thanks to well-established independent segmentation algorithms, and arguably even faster inference. 157 While most above methods discuss registration between 3D MR and 3D transrectal ultrasound images, there have also been advances in aligning 3D MRI to 2D ultrasound which is much easier to obtain.…”

Meta-Learning Initializations for Interactive Medical Image Registration

“…Using methodologies from "classical" iterative registration algorithms, learningbased methods have been proposed. Learning-based methods have used different architectures, such as convolutional neural networks [1][2] and vision transformers [3], different training strategies, such as generative adversarial networks [4,5], supervised [1,4], unsupervised [2,[6][7][8] or reinforcement learning [9][10][11], or different transformation constraints, based on parametric splines [6], diffeomorphism [12] and biomechanics [13]. Semi-supervised learning [14], few-shotand meta-learning [15][16], unsupervised contrastive learning [17], inference-time augmentation [16,18], and amortized hyperparameter learning [19] methodologies have also been used to improve data efficiency and generalizability.…”