Towards near real-time assessment of surgical skills: A comparison of feature extraction techniques

Anh, Nguyen Xuan; Nataraja, Ramesh M.; Chauhan, Sunita

doi:10.1016/j.cmpb.2019.105234

Cited by 45 publications

(63 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The encoder aims to create a copy of its input as an output, reverse engineering the problem by trying to find the right filter. The autoencoder created by Ahn et al [ 34 ] is built using convolutional layers. It is also common to use fully connected layers.…”

Section: Methodsmentioning

confidence: 99%

“…Given the ROI data and the generated output, grouping the data according to the surgical tasks and the expertise level of users constitutes the final output. This is accomplished using the sliding window preprocessing method implemented by Anh et al for their benchmark [ 34 ], to process the multivariate time series and separate chunks of the data into uniformly sized local windows, thereby enabling the evaluation of our data with the same networks originally designed for the kinematic data of JIGSAWS.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery

Lajkó

Elek

Haidegger

2021

Sensors

View full text Add to dashboard Cite

Objective skill assessment-based personal performance feedback is a vital part of surgical training. Either kinematic—acquired through surgical robotic systems, mounted sensors on tooltips or wearable sensors—or visual input data can be employed to perform objective algorithm-driven skill assessment. Kinematic data have been successfully linked with the expertise of surgeons performing Robot-Assisted Minimally Invasive Surgery (RAMIS) procedures, but for traditional, manual Minimally Invasive Surgery (MIS), they are not readily available as a method. 3D visual features-based evaluation methods tend to outperform 2D methods, but their utility is limited and not suited to MIS training, therefore our proposed solution relies on 2D features. The application of additional sensors potentially enhances the performance of either approach. This paper introduces a general 2D image-based solution that enables the creation and application of surgical skill assessment in any training environment. The 2D features were processed using the feature extraction techniques of a previously published benchmark to assess the attainable accuracy. We relied on the JHU–ISI Gesture and Skill Assessment Working Set dataset—co-developed by the Johns Hopkins University and Intuitive Surgical Inc. Using this well-established set gives us the opportunity to comparatively evaluate different feature extraction techniques. The algorithm reached up to 95.74% accuracy in individual trials. The highest mean accuracy—averaged over five cross-validation trials—for the surgical subtask of Knot-Tying was 83.54%, for Needle-Passing 84.23% and for Suturing 81.58%. The proposed method measured well against the state of the art in 2D visual-based skill assessment, with more than 80% accuracy for all three surgical subtasks available in JIGSAWS (Knot-Tying, Suturing and Needle-Passing). By introducing new visual features—such as image-based orientation and image-based collision detection—or, from the evaluation side, utilising other Support Vector Machine kernel methods, tuning the hyperparameters or using other classification methods (e.g., the boosted trees algorithm) instead, classification accuracy can be further improved. We showed the potential use of optical flow as an input for RAMIS skill assessment, highlighting the maximum accuracy achievable with these data by evaluating it with an established skill assessment benchmark, by evaluating its methods independently. The highest performing method, the Residual Neural Network, reached means of 81.89%, 84.23% and 83.54% accuracy for the skills of Suturing, Needle-Passing and Knot-Tying, respectively.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery

Lajkó

Elek

Haidegger

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Some recent techniques compare tool trajectories to "optimal" trajectories [12] or use streamed kinematic data to classify stylistic behavior [13]. Other work has been done to facilitate extraction of the most relevant information during surgery for expertise evaluation, thus reducing the memory and computational effort needed [10]. While these results are promising, advances in the field are still not at a stage at which such information can be integrated in a completely automated robotic loop.…”

Section: Real-time Metrics Of Expertisementioning

confidence: 99%

Rethinking Autonomous Surgery: Focusing on Enhancement over Autonomy

Battaglia

Boehm

Zheng

et al. 2021

European Urology Focus

View full text Add to dashboard Cite

Context: As robot-assisted surgery is increasingly used in surgical care, the engineering research effort towards surgical automation has also increased significantly. Automation promises to enhance surgical outcomes, offload mundane or repetitive tasks, and improve workflow. However, we must ask an important question: should autonomous surgery be our long-term goal? Objective: To provide an overview of the engineering requirements for automating control systems, summarize technical challenges in automated robotic surgery, and review sensing and modeling techniques to capture real-time human behaviors for integration into the robotic control loop for enhanced shared or collaborative control. Evidence acquisition: We performed a nonsystematic search of the English language literature up to March 25, 2021. We included original studies related to automation in robot-assisted laparoscopic surgery and human-centered sensing and modeling. Evidence synthesis: We identified four comprehensive review papers that present techniques for automating portions of surgical tasks. Sixteen studies relate to human-centered sensing technologies and 23 to computer vision and/or advanced artificial intelligence or machine learning methods for skill assessment. Twenty-two studies evaluate or review the role of haptic or adaptive guidance during some learning task, with only a few applied to robotic surgery. Finally, only three studies discuss the role of some form of training in patient outcomes and none evaluated the effects of full or semi-autonomy on patient outcomes. Conclusions: Rather than focusing on autonomy, which eliminates the surgeon from the loop, research centered on more fully understanding the surgeon's behaviors, goals, and limitations could facilitate a superior class of collaborative surgical robots that could be more effective and intelligent than automation alone. Patient summary: We reviewed the literature for studies on automation in surgical robotics and on modeling of human behavior in human-machine interaction. The main application is to enhance the ability of surgical robotic systems to collaborate more effectively and intelligently with human surgeon operators.

show abstract

“…The most convenient hand-engineered features are those introduced by descriptive statistics Anh et al (2020) . In a skill rating system proposed by Brown et al (2016) , eight values of mean, standard deviation, minimum, maximum, range, root-mean-square (RMS), total sum-of-squares (TSS), and time integral of force and acceleration signals are calculated.…”

Section: Data Driven Scoringmentioning

confidence: 99%

Applications of Haptic Technology, Virtual Reality, and Artificial Intelligence in Medical Training During the COVID-19 Pandemic

et al. 2021

View full text Add to dashboard Cite

This paper examines how haptic technology, virtual reality, and artificial intelligence help to reduce the physical contact in medical training during the COVID-19 Pandemic. Notably, any mistake made by the trainees during the education process might lead to undesired complications for the patient. Therefore, training of the medical skills to the trainees have always been a challenging issue for the expert surgeons, and this is even more challenging in pandemics. The current method of surgery training needs the novice surgeons to attend some courses, watch some procedure, and conduct their initial operations under the direct supervision of an expert surgeon. Owing to the requirement of physical contact in this method of medical training, the involved people including the novice and expert surgeons confront a potential risk of infection to the virus. This survey paper reviews recent technological breakthroughs along with new areas in which assistive technologies might provide a viable solution to reduce the physical contact in the medical institutes during the COVID-19 pandemic and similar crises.

show abstract

Towards near real-time assessment of surgical skills: A comparison of feature extraction techniques

Cited by 45 publications

References 23 publications

Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery

Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery

Rethinking Autonomous Surgery: Focusing on Enhancement over Autonomy

Applications of Haptic Technology, Virtual Reality, and Artificial Intelligence in Medical Training During the COVID-19 Pandemic

Contact Info

Product

Resources

About