Deep Learning-Based Haptic Guidance for Surgical Skills Transfer

Abstract

Having a trusted and useful system that helps to diminish the risk of medical errors and facilitate the improvement of quality in the medical education is indispensable. Thousands of surgical errors are occurred annually with high adverse event rate, despite inordinate number of devised patients safety initiatives. Inadvertently or otherwise, surgeons play a critical role in the aforementioned errors. Training surgeons is one of the most crucial and delicate parts of medical education and needs more attention due to its practical intrinsic. In contrast to engineering, dealing with mortal alive creatures provides a minuscule chance of trial and error for trainees. Training in operative rooms, on the other hand, is extremely expensive in terms of not only equipment but also hiring professional trainers. In addition, the COVID-19 pandemic has caused to establish initiatives such as social distancing in order to mitigate the rate of outbreak. This leads surgeons to postpone some non-urgent surgeries or operate with restrictions in terms of safety. Subsequently, educational systems are affected by the limitations due to the pandemic. Skill transfer systems in cooperation with a virtual training environment is thought as a solution to address aforesaid issues. This enables not only novice surgeons to enrich their proficiency but also helps expert surgeons to be supervised during the operation. This paper focuses on devising a solution based on deep leaning algorithms to model the behavior of experts during the operation. In other words, the proposed solution is a skill transfer method that learns professional demonstrations using different effective factors from the body of experts. The trained model then provides a real-time haptic guidance signal for either instructing trainees or supervising expert surgeons. A simulation is utilized to emulate an operating room for femur drilling surgery, which is a common invasive treatment for osteoporosis. This helps us with both collecting the essential data and assessing the obtained models. Experimental results show that the proposed method is capable of emitting guidance force haptic signal with an acceptable error rate.

Toward a Digital Twin for Arthroscopic Knee Surgery: A Systematic Review

“…Kaluschke et al [140] demonstrated a novel algorithm for material removal based on the god-object method and implemented their algorithm on a Kuka LBR robot for 6DOF high-force feedback (up to 200N). Fekri et al [141] used a recursive neural network with LSTM architecture to capture expert behavior during surgical drilling. This behavior was intended for the haptic guidance of novice surgeons during training.…”

Evaluation of surgical skill using machine learning with optimal wearable sensor locations

“…Previously researchers have estimated surgical skills using i)kinematic data and convolutional neural network [ 9 ], ii) kinematic data as putative markers and deep neural networks [ 10 ], iii) virtual reality spinal task and machine learning algorithms (support vector machines, k-nearest neighbors, least discriminant analysis, naïve bayes and decision tree) [ 11 ], iv) image processing and deep neural network during robotic surgery [ 12 – 14 ], v) kinematic data from da Vinci robot and global rating score and machine learning (kNN, logistic regression, SVM) [ 15 ]. Recently deep learning-based haptic guidance systems have been used for surgical skill development [ 16 ]. Moreover, it is unknown which features from wearable sensors such as EMG and accelerometers could contribute more to skill identification.…”