Surgical motion analysis using discriminative interpretable patterns

Forestier, Germain; Petitjean, François; Senin, Pavel; Despinoy, Fabien; Huaulmé, Arnaud; Fawaz, Hassan Ismail; Weber, Jonathan; Idoumghar, Lhassane; Muller, Pierre Alain; Jannin, Pierre

doi:10.1016/j.artmed.2018.08.002

Cited by 56 publications

(65 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hand motion studies have also been done in a simulation environment [9,10]. Similar differences in trajectory analysis were also reported by others [16,19,23,36]. Trajectory analysis in these studies showed results similar to those in the present study for the JIGSAWS data ( Fig.…”

Section: Discussionsupporting

confidence: 89%

“…The JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS) dataset was generated at Johns Hopkins University and is a standardized dataset from simulated RMIS with three exercises (suturing, knot-tying and needle passing) performed by eight participants with varied prior experience [22]. JIGSAWS is the largest publicly available dataset for gesture analysis, and previous work has focused on skill evaluation, gesture classification, gesture segmentation and surgical task recognition [23]. The dataset is fixed and cannot be modified.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set using Robotics Video and Motion Assessment Software

et al. 2020

View full text Add to dashboard Cite

Purpose The JIGSAWS dataset is a fixed dataset of robot-assisted surgery kinematic data used to develop predictive models of skill. The purpose of this study is to analyze the relationships of self-defined skill level with global rating scale scores and kinematic data (time, path length and movements) from three exercises (suturing, knot-tying and needle passing) (right and left hands) in the JIGSAWS dataset. Methods Global rating scale scores are reported in the JIGSAWS dataset and kinematic data were calculated using ROVIMAS software. Self-defined skill levels are in the dataset (novice, intermediate, expert). Correlation coefficients (global rating scale-skill level and global rating scale-kinematic parameters) were calculated. Kinematic parameters were compared among skill levels. Results Global rating scale scores correlated with skill in the knot-tying exercise (r = 0.55, p = 0.0005). In the suturing exercise, time, path length (left) and movements (left) were significantly different (p < 0.05) for novices and experts. For knot-tying, time, path length (right and left) and movements (right) differed significantly for novices and experts. For needle passing, no kinematic parameter was significantly different comparing novices and experts. The only kinematic parameter that correlated with global rating scale scores is time in the knot-tying exercise. Conclusion Global rating scale scores weakly correlate with skill level and kinematic parameters. The ability of kinematic parameters to differentiate among self-defined skill levels is inconsistent. Additional data are needed to enhance the dataset and facilitate subset analyses and future model development.

show abstract

Section: Discussionsupporting

confidence: 89%

Section: Introductionmentioning

confidence: 99%

Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set using Robotics Video and Motion Assessment Software

et al. 2020

View full text Add to dashboard Cite

show abstract

Section: Research Using Ravenmentioning

confidence: 99%

“…Several groups have used Raven in experiments studying measurement or acquisition or updating of surgical skills in robotic surgeons [37][38][39]. For example, [37] quantized robotic gestures into strings which were shown statistically able to discriminate skill level. Another skill assessment study [38] compared performance and learning of trainees between a hands-on user interface device and a contactless control interface based on a low cost (compared to haptic devices) depth sensing camera and hand gestures.…”

Section: Research Using Ravenmentioning

confidence: 99%

“…Simulation is a significant topic in robotic surgery as teleoperated robotic surgeries become more common and drive the need for a cost-efficient way to improve surgeons' skills in operating surgical robots. However, there is only one Raven citation that studies the simulation [61], and only two raven citations are about evaluation [37,67].…”

Section: Simulation and Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

The Raven Open Surgical Robotic Platforms: A Review and Prospect

2019

APH

View full text Add to dashboard Cite

The Raven I and the Raven II surgical robots, as open research platforms, have been serving the robotic surgery research community for ten years. The paper 1) briefly presents the Raven I and the Raven II robots, 2) reviews the recent publications that are built upon the Raven robots, aim to be applied to the Raven robots, or are directly compared with the Raven robots, and 3) uses the Raven robots as a case study to discuss the popular research problems in the research community and the trend of robotic surgery study. Instead of being a thorough literature review, this work only reviews the works formally published in the past three years and uses these recent publications to analyze the research interests, the popular open research problems, and opportunities in the topic of robotic surgery.

show abstract

Evaluation of Deep Learning Models for Identifying Surgical Actions and Measuring Performance

Khalid¹,

Goldenberg²,

Grantcharov³

et al. 2020

JAMA Netw Open

102

View full text Add to dashboard Cite

IMPORTANCE When evaluating surgeons in the operating room, experienced physicians must rely on live or recorded video to assess the surgeon's technical performance, an approach prone to subjectivity and error. Owing to the large number of surgical procedures performed daily, it is infeasible to review every procedure; therefore, there is a tremendous loss of invaluable performance data that would otherwise be useful for improving surgical safety. OBJECTIVE To evaluate a framework for assessing surgical video clips by categorizing them based on the surgical step being performed and the level of the surgeon's competence. DESIGN, SETTING, AND PARTICIPANTS This quality improvement study assessed 103 video clips of 8 surgeons of various levels performing knot tying, suturing, and needle passing from the Johns Hopkins University-Intuitive Surgical Gesture and Skill Assessment Working Set. Data were collected before 2015, and data analysis took place from March to July 2019. MAIN OUTCOMES AND MEASURES Deep learning models were trained to estimate categorical outputs such as performance level (ie, novice, intermediate, and expert) and surgical actions (ie, knot tying, suturing, and needle passing). The efficacy of these models was measured using precision, recall, and model accuracy. RESULTS The provided architectures achieved accuracy in surgical action and performance calculation tasks using only video input. The embedding representation had a mean (root mean square error [RMSE]) precision of 1.00 (0) for suturing, 0.99 (0.01) for knot tying, and 0.91 (0.11) for needle passing, resulting in a mean (RMSE) precision of 0.97 (0.01). Its mean (RMSE) recall was 0.94 (0.08) for suturing, 1.00 (0) for knot tying, and 0.99 (0.01) for needle passing, resulting in a mean (RMSE) recall of 0.98 (0.01). It also estimated scores on the Objected Structured Assessment of Technical Skill Global Rating Scale categories, with a mean (RMSE) precision of 0.85 (0.09) for novice level, 0.67 (0.07) for intermediate level, and 0.79 (0.12) for expert level, resulting in a mean (RMSE) precision of 0.77 (0.04). Its mean (RMSE) recall was 0.85 (0.05) for novice level, 0.69 (0.14) for intermediate level, and 0.80 (0.13) for expert level, resulting in a mean (RMSE) recall of 0.78 (0.03). CONCLUSIONS AND RELEVANCE The proposed models and the accompanying results illustrate that deep machine learning can identify associations in surgical video clips. These are the first steps to creating a feedback mechanism for surgeons that would allow them to learn from their experiences and refine their skills.

show abstract

Surgical motion analysis using discriminative interpretable patterns

Abstract: The proposed approach is an interesting addition to existing learning tools for surgery as it provides a way to obtain a feedback on which parts of an exercise have been used to classify the attempt as correct or incorrect.

Cited by 56 publications

References 54 publications

Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set using Robotics Video and Motion Assessment Software

Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set using Robotics Video and Motion Assessment Software

The Raven Open Surgical Robotic Platforms: A Review and Prospect

Evaluation of Deep Learning Models for Identifying Surgical Actions and Measuring Performance

Contact Info

Product

Resources

About