Learning Based Estimation of 7 DOF Instrument and Grasping Forces on the da Vinci Research Kit

Mixed reality based teleoperation and visualization of surgical robotics

“…Chen et al. [ 11 ] modified the previous method by implementing hand tracking provided by HoloLens 2 to control the robot's pose. They applied the relative translation and orientation of the tracking hand to those of the robot and used hand gestures to engage with the robot, minimizing unintentional hand movements.…”

“…In Chen et al. 's work [ 11 ], they implemented hand gestures to teleoperate the instrument mounted on the PSM and achieved comparable performance with the conventional method in the virtual ring‐wire task. However, their implementation did not integrate the endoscope and only considered the motion of the instrument with respect to the world coordinate system, which was determined by the starting position of the HoloLens.…”

“…While the dedicated console of the da Vinci surgical system offers valuable features like fine motion scaling/filtering, instrument control, and 3D visualization [ 9 ], it unavoidably creates physical barriers between the surgeon and the patient. This obstruction of direct sight to the patient, coupled with the restricted mobility imposed on the surgeon, diminishes their awareness of the patient's condition and may potentially hinder surgical efficiency and safety [ 10 , 11 ]. Additionally, the bulky nature of the console occupies a fixed space, making it immovable and impractical for surgeons to perform surgeries beyond the confines of the operating room.…”