Automatic Markerless Registration and Tracking of the Bone for Computer-Assisted Orthopaedic Surgery

Abstract

To achieve a simple and less invasive registration procedure in computer-assisted orthopaedic surgery, we propose an automatic, markerless registration and tracking method based on depth imaging and deep learning. A depth camera is used to continuously capture RGB and depth images of the exposed bone during surgery, and deep neural networks are trained to first localise the surgical target using the RGB image, then segment the target area of the corresponding depth image, from which the surface geometry of the target bone can be extracted. The extracted surface is then compared to a pre-operative model of the same bone for registration. This process can be performed dynamically during the procedure at a rate of 5-6 Hz, without any need for surgeon intervention or invasive optical markers. Ex vivo registration experiments were performed on a cadaveric knee, and accuracy measurements against an optically tracked ground truth resulted in a mean translational error of 2.74 mm and a mean rotational error of 6.66 •. Our results are the first to describe a promising new way to achieve automatic markerless registration and tracking in computerassisted orthopaedic surgery, demonstrating that truly seamless registration and tracking of the limb is within reach. Our method reduces invasiveness by removing the need for percutaneous markers. The surgeon is also exempted from inserting markers and collecting registration points manually, which contributes to a more efficient surgical workflow and shorter procedure time in the operating room. INDEX TERMS Computer-assisted orthopaedic surgery, deep learning, depth imaging, markerless registration.

Markerless Navigation System for Orthopaedic Knee Surgery: A Proof of Concept Study

“…Neural networks can be trained to "learn" comprehensive semantics using labelled dataset. A proof-of-concept study was outlined in [14], and a more systematic validation was provided by [13]. Both works used a consumer-level RGB-D camera to record surgical site.…”

“…Two sequential convolutional neural networks were trained to segment the target anatomy from real-time camera captures automatically. As shown in Figure 2 (he detailed network architecture can be found in [13]), the knee was first localised in the RGB capture by a pre-trained localisation network. According to the inferred region of interest (ROI), the depth frame was cropped to remove the irrelevant background.…”

“…Automatic markerless target tracking and registration are desired to reduce such risks and encourage the broader acceptance of CAOS systems. Markerless tracking and registration algorithm has been proposed in [13], [14] for knee surgeries. A deep neural network was trained to segment the target femur from the RGB-D capture of surgical site.…”

A Survey of Marker-Less Tracking and Registration Techniques for Health & Environmental Applications to Augmented Reality and Ubiquitous Geospatial Information Systems

“…This occurs when an object is occluded by some part of itself (self-occlusion) or another object (external occlusion). Self-occlusion can be solved by computing a table of visible features [7] or depth buffering [126]. Using outlier detection algorithms such as RANSAC can address external occlusion even in highly cluttered environments with a low inlier information ratio [94].…”

Automatic Markerless Tracking for Computer-Assisted Orthopaedic Surgery

“…Instead of performing a manual initial registration and updating the transformation based on marker tracking results, markerless tracking can be automatically achieved by a fast registration between a reference model and the real-time segmented target in world space. One of our previous researches has shown that a trained neural network can successfully segment the target femur from a RGBD video, with intersection over union of up to 0.87 [3]. The aim of this paper is to complete the markerless tracking process by applying an accurate and fast registration algorithm to these segmentation results.…”