Pose-dependent weights and Domain Randomization for fully automatic X-ray to CT Registration

Abstract

Fully automatic X-ray to CT registration requires a solid initialization to provide an initial alignment within the capture range of existing intensity-based registrations. This work adresses that need by providing a novel automatic initialization, which enables end to end registration. First, a neural network is trained once to detect a set of anatomical landmarks on simulated X-rays. A domain randomization scheme is proposed to enable the network to overcome the challenge of being trained purely on simulated data and run inference on real Xrays. Then, for each patient CT, a patient-specific landmark extraction scheme is used. It is based on backprojecting and clustering the previously trained network's predictions on a set of simulated X-rays. Next, the network is retrained to detect the new landmarks. Finally the combination of network and 3D landmark locations is used to compute the initialization using a perspective-n-point algorithm. During the computation of the pose, a weighting scheme is introduced to incorporate the confidence of the network in detecting the landmarks. The algorithm is evaluated on the pelvis using both real and simulated x-rays. The mean (± standard deviation) target registration error in millimetres is 4.1 ± 4.3 for simulated X-rays with a success rate of 92% and 4.2 ± 3.9 for real X-rays with a success rate of 86.8%, where a success is defined as a translation error of less than 30mm.

Deep Iterative 2D/3D Registration

“…Use of simulated data in combination with domain randomization [3] or adversarial data augmentation [14] strategies can be explored to reduce the burden of annotated training data requirements.…”

“…Deep Learning (DL) based techniques for 2D/3D registration problem have shown promising results, by improving the computational efficiency [11,17] and robustness [10,8,19]. Recent works have also shown that the robustness can be increased to a much greater extent using DL-based methods and even propose fully automatic 2D/3D registration solution [2,3]. However, most of these techniques rely on a final refinement step based on the classical methods to achieve the necessary registration accuracy for interventional application, which limits the computational efficiency of DL-based registration.…”