Facial landmark‐guided surface matching for image‐to‐patient registration with an RGB‐D camera

Abstract

Background: Fiducial marker-based image-to-patient registration is the most common way in image-guided neurosurgery, which is labour-intensive, time consuming, invasive and error prone. Methods:We proposed a method of facial landmark-guided surface matching for image-to-patient registration using an RGB-D camera. Five facial landmarks are localised from preoperative magnetic resonance (MR) images using deep learning and RGB image using Adaboost with multi-scale block local binary patterns, respectively. The registration of two facial surface point clouds derived from MR images and RGB-D data is initialised by aligning these five landmarks and further refined by weighted iterative closest point algorithm.Results: Phantom experiment results show the target registration error is less than 3 mm when the distance from the camera to the phantom is less than 1000 mm. The registration takes less than 10 s. Conclusions:The proposed method is comparable to the state-of-the-arts in terms of the accuracy yet more time-saving and non-invasive.

Image-to-Patient Registration in Computer-Assisted Surgery of Head and Neck: State-of-the-Art, Perspectives, and Challenges

“…In the field of image-to-patient registration and in contrast with other medical imaging domains, deep learning is not common [ 139 ]. So far, it cannot compete with the state-of-the-art methods [ 28 , 140 ], but will certainly improve in the coming years. Obstacles in this domain are numerous, and among these, the most-prominent are the lack of large databanks, anatomical variations, scenario variations, and noisy images due to the disease or bleeding.…”

Use of Neuronavigation and Augmented Reality in Transsphenoidal Pituitary Adenoma Surgery

“…To rely on AR throughout surgery, especially when performing surgery close to vascular risk structures within a restricted space such as in transnasal transsphenoidal surgery with a limited line of sight, high navigational accuracy is a prerequisite. Standard fiducial- or landmark-based registration approaches as most commonly used [ 37 ] are heavily user dependent including the placement of artificial markers (location and amount of markers) [ 38 ] before imaging and the intraoperative acquisition of those landmarks using the pointer (e.g., skin shift [ 39 ]). In this study TRE (1.85 ± 1.02 mm) for fiducial-based registration ranged from 0.55 mm to 3.43 mm, which is comparable to previous studies of our group [ 13 , 40 ].…”

LocalEyenet: Deep Attention Framework for Localization of Eyes

“…al presented a Constrained Local Neural Field (CLNF) [18] that take care of the feature detection problem in wild scenario with less illumination and blurred faces. Recently Weighted Iterative Closest Point (ICP) based surface matching [19] and hierarchical filtering strategy [20] have been used to reduce the effect of noise in face registration. Researchers have recently proposed a semi-supervised method [21], called self-calibrated pose attention network (SCPAN), that computes Boundary-Aware Landmark Intensity (BALI) fields corresponding to a boundary and the landmarks closest to the boundary.…”