Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set using Robotics Video and Motion Assessment Software

Lefor, Alan Kawarai; Harada, Kensuke; Dosis, A.; Mitsuishi, Mamoru

doi:10.1007/s11548-020-02259-z

Cited by 19 publications

(16 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Funke et al’s hybrid 3D network misclassified every intermediate surgeon into either expert or novice, using OF data for Knot-Tying [ 25 ]. This may partially be due to data disparity, as intermediate and expert subjects are underrepresented in comparison to novices [ 29 ]. Funke’s solution relied on a Leave-One-User-Out (LOUO) cross-validation, which is less robust to such disparity.…”

Section: Resultsmentioning

confidence: 99%

“…Created by the cooperation of the Johns Hopkins University (JHU) and the creators of the da Vinci Surgical System [ 14 ], Intuitive Surgical, Inc. (ISI), the JHU–ISI Gesture and Skill Assessment Working Set (JIGSAWS) [ 29 ] is one of the most widely used surgical skill assessment datasets—as evidenced by Yanik et al [ 21 ]. It is also the only publicly available annotated dataset for surgical skill assessment.…”

Section: Methodsmentioning

confidence: 99%

“…)—to perform classification. Taking the kinematic data of the JIGSAWS set as input [ 29 ], they implemented 9 different evaluation methods to provide a benchmark for skill assessment solutions, categorising surgical skills into expert, intermediate and novice classes. Our solution uses some of the same methods from this codebase, repurposed for 2D visual input features.…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery

Lajkó

Elek

Haidegger

2021

Sensors

View full text Add to dashboard Cite

Objective skill assessment-based personal performance feedback is a vital part of surgical training. Either kinematic—acquired through surgical robotic systems, mounted sensors on tooltips or wearable sensors—or visual input data can be employed to perform objective algorithm-driven skill assessment. Kinematic data have been successfully linked with the expertise of surgeons performing Robot-Assisted Minimally Invasive Surgery (RAMIS) procedures, but for traditional, manual Minimally Invasive Surgery (MIS), they are not readily available as a method. 3D visual features-based evaluation methods tend to outperform 2D methods, but their utility is limited and not suited to MIS training, therefore our proposed solution relies on 2D features. The application of additional sensors potentially enhances the performance of either approach. This paper introduces a general 2D image-based solution that enables the creation and application of surgical skill assessment in any training environment. The 2D features were processed using the feature extraction techniques of a previously published benchmark to assess the attainable accuracy. We relied on the JHU–ISI Gesture and Skill Assessment Working Set dataset—co-developed by the Johns Hopkins University and Intuitive Surgical Inc. Using this well-established set gives us the opportunity to comparatively evaluate different feature extraction techniques. The algorithm reached up to 95.74% accuracy in individual trials. The highest mean accuracy—averaged over five cross-validation trials—for the surgical subtask of Knot-Tying was 83.54%, for Needle-Passing 84.23% and for Suturing 81.58%. The proposed method measured well against the state of the art in 2D visual-based skill assessment, with more than 80% accuracy for all three surgical subtasks available in JIGSAWS (Knot-Tying, Suturing and Needle-Passing). By introducing new visual features—such as image-based orientation and image-based collision detection—or, from the evaluation side, utilising other Support Vector Machine kernel methods, tuning the hyperparameters or using other classification methods (e.g., the boosted trees algorithm) instead, classification accuracy can be further improved. We showed the potential use of optical flow as an input for RAMIS skill assessment, highlighting the maximum accuracy achievable with these data by evaluating it with an established skill assessment benchmark, by evaluating its methods independently. The highest performing method, the Residual Neural Network, reached means of 81.89%, 84.23% and 83.54% accuracy for the skills of Suturing, Needle-Passing and Knot-Tying, respectively.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery

Lajkó

Elek

Haidegger

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Together with time features like task completion time, these values are used as inputs for a random forest classifier to rate the peg transfer score of 38 different participants. In Javaux et al (2018) , metrics like mean/maximum velocity and acceleration, tool path length, depth perception, maximum and integral of planar/vertical force, and task completion time are considered as a baseline for skill assessment Lefor et al (2020) . Another commonly used method in the literature is to use statistical tests such as Mann-Whitney test Moody et al (2008) , Kruskal–Wallis test Javaux et al (2018) , Pearson or Spearman correlation Zendejas et al (2017) , etc.…”

Section: Data Driven Scoringmentioning

confidence: 99%

Applications of Haptic Technology, Virtual Reality, and Artificial Intelligence in Medical Training During the COVID-19 Pandemic

et al. 2021

View full text Add to dashboard Cite

This paper examines how haptic technology, virtual reality, and artificial intelligence help to reduce the physical contact in medical training during the COVID-19 Pandemic. Notably, any mistake made by the trainees during the education process might lead to undesired complications for the patient. Therefore, training of the medical skills to the trainees have always been a challenging issue for the expert surgeons, and this is even more challenging in pandemics. The current method of surgery training needs the novice surgeons to attend some courses, watch some procedure, and conduct their initial operations under the direct supervision of an expert surgeon. Owing to the requirement of physical contact in this method of medical training, the involved people including the novice and expert surgeons confront a potential risk of infection to the virus. This survey paper reviews recent technological breakthroughs along with new areas in which assistive technologies might provide a viable solution to reduce the physical contact in the medical institutes during the COVID-19 pandemic and similar crises.

show abstract

“…Created by the cooperation of the Johns Hopkins University (JHU) and the creators of the Da Vinci Surgical System [11], Intuitive Surgical, Inc. (ISI) -the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS) [12] is one of the most widely-used surgical skill assessment datasets -as evidenced by [9] with expert rating scores by an experienced surgeon [13].…”

Section: A Jigsawsmentioning

confidence: 99%

Surgical Skill Assessment Automation Based on Sparse Optical Flow Data

Lajkó¹,

Elek

Haidegger

2021

2021 IEEE 25th International Conference on Intelligent Engineering Systems (INES)

View full text Add to dashboard Cite

Objective skill assessment based personal feedback is a vital part of surgical training. Automated assessment solutions aim to replace traditional manual (experts' opinion-based) assessment techniques, that predominantly requires the most valuable time commitment from senior surgeons. Typically, either kinematic or visual input data can be employed to perform skill assessment. Minimally Invasive Surgery (MIS) benefits the patients by using smaller incisions than open surgery, resulting in less pain and quicker recovery, but increasing the difficulty of the surgical task manyfold. Robot-Assisted Minimally Invasive Surgery (RAMIS) offers higher precision during surgery, while also improving the ergonomics for the performing surgeons. Kinematic data have been proven to directly correlate with the expertise of surgeons performing RAMIS procedures, but for traditional MIS it is not readily available. Visual feature-based solutions are slowly catching up to the efficacy of kinematicsbased solutions, but the best performing methods usually depend on 3D visual features, which require stereo cameras and calibration data, neither of which are available in MIS. This paper introduces a general 2D image-based solution that can enable the creation and application of surgical skill assessment solutions in any training environment. A well-established kinematics-based skill assessment benchmark's feature extraction techniques have been repurposed to evaluate the accuracy that the generated data can produce. We reached individual accuracy up to 95.74% and mean accuracy -averaged over 5 cross-validation trialsup to 83.54%. Additional related resources such as the source codes, result and data files are publicly available on Github (https://github.com/ABC-iRobotics/VisDataSurgicalSkill).

show abstract

Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set using Robotics Video and Motion Assessment Software

Cited by 19 publications

References 28 publications

Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery

Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery

Applications of Haptic Technology, Virtual Reality, and Artificial Intelligence in Medical Training During the COVID-19 Pandemic

Surgical Skill Assessment Automation Based on Sparse Optical Flow Data

Contact Info

Product

Resources

About