An Augmented Reality-Based Interaction Scheme for Robotic Pedicle Screw Placement

Abstract

Robot-assisted surgery is becoming popular in the operation room (OR) for, e.g., orthopedic surgery (among other surgeries). However, robotic executions related to surgical steps cannot simply rely on preoperative plans. Using pedicle screw placement as an example, extra adjustments are needed to adapt to the intraoperative changes when the preoperative planning is outdated. During surgery, adjusting a surgical plan is non-trivial and typically rather complex since the available interfaces used in current robotic systems are not always intuitive to use. Recently, thanks to technical advancements in head-mounted displays (HMD), augmented reality (AR)-based medical applications are emerging in the OR. The rendered virtual objects can be overlapped with real-world physical objects to offer intuitive displays of the surgical sites and anatomy. Moreover, the potential of combining AR with robotics is even more promising; however, it has not been fully exploited. In this paper, an innovative AR-based robotic approach is proposed and its technical feasibility in simulated pedicle screw placement is demonstrated. An approach for spatial calibration between the robot and HoloLens 2 without using an external 3D tracking system is proposed. The developed system offers an intuitive AR–robot interaction approach between the surgeon and the surgical robot by projecting the current surgical plan to the surgeon for fine-tuning and transferring the updated surgical plan immediately back to the robot side for execution. A series of bench-top experiments were conducted to evaluate system accuracy and human-related errors. A mean calibration error of 3.61 mm was found. The overall target pose error was 3.05 mm in translation and 1.12∘ in orientation. The average execution time for defining a target entry point intraoperatively was 26.56 s. This work offers an intuitive AR-based robotic approach, which could facilitate robotic technology in the OR and boost synergy between AR and robots for other medical applications.

Development and evaluation of robot‐assisted ultrasound navigation system for pedicle screw placement: An ex‐vivo animal validation

“…The following approach was proposed to qualify the screw trajectory's accuracy. When evaluating the system's positioning accuracy, the Z coordinate was not considered because the screw would move along the trajectory 24 . The orientation around the Z axis of the screw trajectory was also not measured since the screw was able to rotate around its Z axis.…”

“…When evaluating the system's positioning accuracy, the Z coordinate was not considered because the screw would move along the trajectory. 24 The orientation around the Z axis of the screw trajectory was also not measured since the screw was able to rotate around its Z axis. The position and orientation errors were calculated by comparing the projected screw trajectory R p 00 EP with the targeted trajectory R p EP obtained from optical navigation, as shown in e r was calculated as a root sum square, as defined in the previous study.…”

Assessing the Accuracy of Spinal Instrumentation Using Augmented Reality (AR): A Systematic Review of the Literature and Meta-Analysis

“…While these adjuncts may have different ideal applications, currently, the decision on which adjunct to use should be made based on surgeon comfort. The recent literature, including the study by Vörös et al, has explored a mixed-model approach, combining the strengths of both AR and robotic systems [18].…”

Ultrasound-Based Robot-Assisted Drilling for Minimally Invasive Pedicle Screw Placement

“…1) Positioning Error: The errors of the entry point 𝑒 𝑒 𝑝 and stop point 𝑒 𝑠 𝑝 are calculated as the shortest distance between the measured entry point and the line connecting the planned entry and destination points, as described in [33]. When evaluating the system positioning accuracy, the rotation around 𝒛 𝑒 𝑝 -axis is not considered because the screw would move along the trajectory [36]. The orientation around the 𝒛 𝑒 𝑝 -axis of the screw trajectory is also not measured since the screw is axisymmetric and this roll angle is not clinically relevant.…”