Automated robot‐assisted surgical skill evaluation: Predictive analytics approach

Fard, Mahtab Jahanbani; Ameri, Sattar; Ellis, R. Darin; Chinnam, Ratna Babu; Pandya, Abhilash; Klein, Michael D.

doi:10.1002/rcs.1850

Cited by 144 publications

(135 citation statements)

References 47 publications

Supporting

Mentioning

131

Contrasting

Order By: Relevance

“…However, this approach involves manual feature engineering, requires task-specific knowledge and significant effort to design optimal skill metrics [23]. In fact, defining the best metrics to capture adequate information and be generalized enough to apply across different types of surgery or groups of surgeons remains an open problem [16,17,24,25].…”

Section: Previous Approaches In Objective Skill Assessmentmentioning

confidence: 99%

“…In this study, we use the self-proclaimed skill levels and GRS-based skill levels as the ground-truth labels for each surgical trial, respectively. In order to label surgeons skill levels using GRS scores, inspired from [24], thresholds of 15 and 20 are used to divide surgeons into novice, intermediate, and expert, in tasks of Needle-passing and Knot-tying, and thresholds of 19 and 24 are used in Suturing for skill labeling.…”

Section: Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery

2018

View full text Add to dashboard Cite

Purpose: With the advent of robot-assisted surgery, the role of data-driven approaches to integrate statistics and machine learning is growing rapidly with prominent interests in objective surgical skill assessment. However, most existing work requires translating robot motion kinematics into intermediate features or gesture segments that are expensive to extract, lack efficiency, and require significant domainspecific knowledge. Methods: We propose an analytical deep learning framework for skill assessment in surgical training. A deep convolutional neural network is implemented to map multivariate time series data of the motion kinematics to individual skill levels.Results: We perform experiments on the public minimally invasive surgical robotic dataset, JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS). Our proposed learning model achieved a competitive accuracy of 92.5%, 95.4%, and 91.3%, in the standard training tasks: Suturing, Needle-passing, and Knot-tying, respectively. Without the need of engineered features or carefully-tuned gesture segmentation, our model can successfully decode skill information from raw motion profiles via end-to-end learning. Meanwhile, the proposed model is able to reliably interpret skills within 1-3 second window, without needing an observation of entire training trial. Conclusion:This study highlights the potentials of deep architectures for an proficient online skill assessment in modern surgical training.

show abstract

Section: Previous Approaches In Objective Skill Assessmentmentioning

confidence: 99%

Section: Datasetmentioning

confidence: 99%

Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery

2018

View full text Add to dashboard Cite

show abstract

“…One approach for OCASE-T is to calculate descriptive metrics from the observed data and to use these metrics to determine surgical skill. For example, Fard et al [10] record instrument trajectories and extract several features from the trajectories, such as path length, motion smoothness, curvature, and task completion time. Using these features, they train various machine learning classifiers to distinguish between expert and novice.…”

mentioning

confidence: 99%

Video-based surgical skill assessment using 3D convolutional neural networks

et al. 2019

View full text Add to dashboard Cite

Purpose: A profound education of novice surgeons is crucial to ensure that surgical interventions are effective and safe. One important aspect is the teaching of technical skills for minimally invasive or robot-assisted procedures. This includes the objective and preferably automatic assessment of surgical skill.Recent studies presented good results for automatic, objective skill evaluation by collecting and analyzing motion data such as trajectories of surgical instruments. However, obtaining the motion data generally requires additional equipment for instrument tracking or the availability of a robotic surgery system to capture kinematic data. In contrast, we investigate a method for automatic, objective skill assessment that requires video data only. This has the advantage that video can be collected effortlessly during minimally invasive and robot-assisted training scenarios. Methods: Our method builds on recent advances in deep learning-based video classification. Specifically, we propose to use an inflated 3D ConvNet to classify snippets, i.e., stacks of a few consecutive frames, extracted from surgical video. The network is extended into a Temporal Segment Network during training. Results: We evaluate the method on the publicly available JIGSAWS dataset, which consists of recordings of basic robot-assisted surgery tasks performed on a dry lab bench-top model. Our approach achieves high skill classification accuracies ranging from 95.1% to 100.0%. Conclusions: Our results demonstrate the feasibility of deep learning-based assessment of technical skill from surgical video. Notably, the 3D ConvNet is able to learn meaningful patterns directly from the data, alleviating the need for man-

show abstract

“…To simplify this process, three metrics are selected to form the baseline skill assessment in this paper. These include the time-normalized instrument path lengthv of the slave robot end-effectors, the number of clutching n c on the master side and the number of failure n f (e.g., the number of dropping the peg or rubber ring during transfer process), which have been reported to demonstrate some aspects of the surgeon's expertise in [11].…”

Section: Skill Level Awarenessmentioning

confidence: 99%

A Self-Adaptive Motion Scaling Framework for Surgical Robot Remote Control

Zhang

Xiao

Huang

et al. 2019

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Master-slave control is a common form of humanrobot interaction for robotic surgery. To ensure seamless and intuitive control, a mechanism of self-adaptive motion scaling during teleoperaton is proposed in this paper. The operator can retain precise control when conducting delicate or complex manipulation, while the movement to a remote target is accelerated via adaptive motion scaling. The proposed framework consists of three components: 1) situation awareness, 2) skill level awareness, and 3) task awareness. The self-adaptive motion scaling ratio allows the operators to perform surgical tasks with high efficiency, forgoing the need of frequent clutching and instrument repositioning. The proposed framework has been verified on a da Vinci Research Kit (dVRK) to assess its usability and robustness. An in-house database is constructed for offline model training and parameter estimation, including both the kinematic data obtained from the robot and visual cues captured through the endoscope. Detailed user studies indicate that a suitable motion-scaling ratio can be obtained and adjusted online. The overall performance of the operators in terms of control efficiency and task completion is significantly improved with the proposed framework.Index Terms-Learning and adaptive systems, telerobotics and teleoperation, medical robots and systems.

show abstract

Automated robot‐assisted surgical skill evaluation: Predictive analytics approach

Abstract: Background: Surgical skill assessment has predominantly been a subjective task. Recently,

Cited by 144 publications

References 47 publications

Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery

Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery

Video-based surgical skill assessment using 3D convolutional neural networks

A Self-Adaptive Motion Scaling Framework for Surgical Robot Remote Control

Contact Info

Product

Resources

About