Automatic Microsurgical Skill Assessment Based on Cross-Domain Transfer Learning

Zhang, Dandan; Wu, Zicong; Chen, Junhong; Gao, Anzhu; Chen, Xu; Li, Peichao; Wang, Zhaoyang; Yang, Guitao; Lo, Benny

doi:10.1109/lra.2020.2989075

Cited by 42 publications

(19 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They also perform experiments in the well-known robotic surgery dataset JIG-SAWS to demonstrate the generalization of their approach, with competitive results. Zhang et al [87] proposed an automatic microsurgical skill assessment for robot-assisted microsurgery based on cross-domain transfer learning. The pretrained model is obtained via the JIGSAWS dataset and then transfered for microsurgical skill assessment.…”

Section: Surgical Skill Assessmentmentioning

confidence: 99%

A Review on Deep Learning in Minimally Invasive Surgery

et al. 2021

View full text Add to dashboard Cite

In the last five years, deep learning has attracted great interest in computer-assisted systems for Minimally Invasive Surgery. The straightforward accessibility to images in surgical interventions makes deep neural networks enormously powerful for solving classification problems in complex surgical scenarios. The objective of this work is to provide readers a survey on deep learning models applied to minimally invasive surgery, identifying the different architectures used depending on the application, the results achieved until now, and the publicly available surgical datasets that can be used for validating new studies. A total of 85 publications have been extracted from manual research from four databases (IEEE Xplorer, Springer Link, Science Direct, and ACM Digital Library). After analyzing all these studies, they have been classified into four applications: surgical image analysis, surgical task analysis, surgical skill assessment, and automation of surgical tasks. This work provides a technical description of these works and a comparison among them. Finally, promising research directions to advance in this field are identified.

show abstract

Section: Surgical Skill Assessmentmentioning

confidence: 99%

A Review on Deep Learning in Minimally Invasive Surgery

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The development of assessment tools for Robot-Assisted Microsurgery (RAMS) skills is still in progress [26][27][28][29]. All of these innovative grading tools are comprehensive and reliable for assessing the students' progress throughout a microsurgical course [30,31].…”

Section: Objective Assessment Tools For Microsurgical Trainingmentioning

confidence: 99%

Microsurgical education in Greece: past, present, and future

et al. 2021

View full text Add to dashboard Cite

The aim of this paper is to provide a brief overview of the history of microsurgery in Greece and how it evolved throughout the years. It is based on published literature as well as anecdotal evidence. It is by no means an exhaustive list of available resources and contributions. Microsurgery in Greece begins with Prof Soucacos who acquired his microsurgical skills in the USA (1970–1974), where he worked as a clinical and research fellow. After gaining invaluable experience, he returned to his home country, Greece, to establish a microsurgery replantation team in 1975. His team gained national recognition soon thereafter thanks to the many successes and innovations they achieved. The tradition is continued with contemporary microsurgical courses in Greece from expert faculty and a busy microsurgical practice in several centers across the country. The experimental educational program in microsurgery includes a blend of synthetic and live animal models, such as rats and rabbits. They include a complete exposure to basic and advanced practical exercises through several days. The simulation training models slowly but surely steadily advance to meet the training standards.

show abstract

“…Kinematic solutions are more commonly used [ 21 ], but they cannot be used for the training of traditional, manual MIS. Given the popularity of deep learning methods in this field of research, the issue of overfitting and thereby the lack of generalisation capability is also an ever-present issue, often combated by cross-validation, regularisation or transfer learning [ 22 ].…”

Section: Introductionmentioning

confidence: 99%

Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery

Lajkó

Elek

Haidegger

2021

Sensors

View full text Add to dashboard Cite

Objective skill assessment-based personal performance feedback is a vital part of surgical training. Either kinematic—acquired through surgical robotic systems, mounted sensors on tooltips or wearable sensors—or visual input data can be employed to perform objective algorithm-driven skill assessment. Kinematic data have been successfully linked with the expertise of surgeons performing Robot-Assisted Minimally Invasive Surgery (RAMIS) procedures, but for traditional, manual Minimally Invasive Surgery (MIS), they are not readily available as a method. 3D visual features-based evaluation methods tend to outperform 2D methods, but their utility is limited and not suited to MIS training, therefore our proposed solution relies on 2D features. The application of additional sensors potentially enhances the performance of either approach. This paper introduces a general 2D image-based solution that enables the creation and application of surgical skill assessment in any training environment. The 2D features were processed using the feature extraction techniques of a previously published benchmark to assess the attainable accuracy. We relied on the JHU–ISI Gesture and Skill Assessment Working Set dataset—co-developed by the Johns Hopkins University and Intuitive Surgical Inc. Using this well-established set gives us the opportunity to comparatively evaluate different feature extraction techniques. The algorithm reached up to 95.74% accuracy in individual trials. The highest mean accuracy—averaged over five cross-validation trials—for the surgical subtask of Knot-Tying was 83.54%, for Needle-Passing 84.23% and for Suturing 81.58%. The proposed method measured well against the state of the art in 2D visual-based skill assessment, with more than 80% accuracy for all three surgical subtasks available in JIGSAWS (Knot-Tying, Suturing and Needle-Passing). By introducing new visual features—such as image-based orientation and image-based collision detection—or, from the evaluation side, utilising other Support Vector Machine kernel methods, tuning the hyperparameters or using other classification methods (e.g., the boosted trees algorithm) instead, classification accuracy can be further improved. We showed the potential use of optical flow as an input for RAMIS skill assessment, highlighting the maximum accuracy achievable with these data by evaluating it with an established skill assessment benchmark, by evaluating its methods independently. The highest performing method, the Residual Neural Network, reached means of 81.89%, 84.23% and 83.54% accuracy for the skills of Suturing, Needle-Passing and Knot-Tying, respectively.

show abstract

Automatic Microsurgical Skill Assessment Based on Cross-Domain Transfer Learning

Cited by 42 publications

References 43 publications

A Review on Deep Learning in Minimally Invasive Surgery

A Review on Deep Learning in Minimally Invasive Surgery

Microsurgical education in Greece: past, present, and future

Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery

Contact Info

Product

Resources

About