Marker-Based Registration for Large Deformations - Application to Open Liver Surgery

Abstract

This paper introduces an Augmented Reality (AR) system for open liver surgery. Although open surgery remains the gold-standard for the treatment of complex tumors and central lesions, technological issues actually prevent using AR with sufficient accuracy for clinical use. We propose a markersbased method allowing for the tracking and the deformation of a preoperative model in real-time during the surgery. Markers are manually placed on the surface of the organ after opening the abdominal cavity, and tracked in real-time by a set of infrared cameras. Our framework is composed of both a nonrigid initial registration method, providing an estimation of the location of the markers in the preoperative model, and a realtime tracking algorithm to deform the model during the surgery (even for large deformation or partial occlusion of the organ). The method is validated on both synthetic and ex-vivo samples; in addition, we demonstrate its applicability in the operating room during a liver resection surgery on a human patient. Preliminary studies provided promising results to improve the location of tumors, and to help surgeons into planning the ideal resection intraoperatively.

Robotic Insertion of Flexible Needle in Deformable Structures Using Inverse Finite-Element Simulation

“…In order to bring the system to the operating room the main limitation lies in the image modality and image processing issues providing reliable per-operative observation points, nec-essary for the corrective step. We are currently considering the possibility to combine this work with our preliminary study [46] where similar markers were used for Augmented Reality during an open liver surgery.…”

“…In the rest of the paper we assume that a sparse set of 3D observation points m located of foam's surface are visible from an external tracking system. This solution has recently been used for AR of open liver surgery [46]. The accuracy of the registration step is not detailed in this paper (see [46], [44] and [38] for more information).…”

Shared control strategy for needle insertion into deformable tissue using inverse Finite Element simulation

“…In practice, fiducials may be inserted percutaneously before the CT-scan acquisition (later used for the segmentation), as performed by [32] to guide the Cyberknife. The registration of the model is then performed with 2D projective constraints as introduced in [33]: the markers are only constrained along the projective lines coming from the center of the projection of the C-ARM as shown in Fig. 2 Therefore the displacement of markers is prescribed in the 2D plane of the projected view but completely free in the orthogonal direction.…”

CBCT‐based navigation system for open liver surgery: Accurate guidance toward mobile and deformable targets with a semi‐rigid organ approximation and electromagnetic tracking of the liver

“…In the first group (i), surface-based registration methods (e.g., based on an optical scan or landmark registration) are utilized to guarantee alignment between the surgical scene and the preoperative plan. [18][19][20][21][22] This approach is largely applied in laparoscopic and robotic surgery, and shows the biggest potential for future integration into the augmented reality (AR) setup. [23][24][25] However, because it does not allow for visualizing the liver's underlying anatomy, accurate registration is only feasible prior to the start of the dissection (e.g., when the complete surface is intact) and its accuracy assessment is primarily based on evaluating liver surface alignment.…”

“…In the first group (i), surface‐based registration methods (e.g., based on an optical scan or landmark registration) are utilized to guarantee alignment between the surgical scene and the preoperative plan 18–22 . This approach is largely applied in laparoscopic and robotic surgery, and shows the biggest potential for future integration into the augmented reality (AR) setup 23–25 .…”