ESAD: Endoscopic Surgeon Action Detection Dataset

Bawa, Vivek Singh; Singh, Gurkirt; KapingA, Francis; Skarga-Bandurova, Inna; Leporini, Alice; Landolfo, Carmela; Stabile, Armando; Setti, Francesco; Muradore, Riccardo; Oleari, Elettra; Cuzzolin, Fabio

doi:10.48550/arxiv.2006.07164

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite recent advances, the field and applicability to real surgical scenarios have been greatly hindered by the lack of large and diverse datasets of annotated demonstrations, which are essential for robust training of modern recognition systems based on machine learning and deep learning. Current methods need testing on varied surgical tasks and procedures, as well as on data from real interventions with complex environments, blood, specularities, camera motions, illumination changes, occlusions and higher variability in motion and gesture ordering [60,87,88]. International collaborations for clinical data collection and sharing would not only accelerate the generation of such datasets, but also allow the representation of variations in human physiology linked to ethnicity as well as a wider spectrum of surgical techniques.…”

Section: Translational Researchmentioning

confidence: 99%

Gesture Recognition in Robotic Surgery: A Review

Amsterdam

Clarkson²,

Stoyanov³

2021

IEEE Trans. Biomed. Eng.

View full text Add to dashboard Cite

Surgical activity recognition is a fundamental step in computer-assisted interventions. This paper reviews the state-of-the-art in methods for automatic recognition of fine-grained gestures in robotic surgery focusing on recent data-driven approaches and outlines the open questions and future research directions. Methods: An article search was performed on 5 bibliographic databases with combinations of the following search terms: robotic, robot-assisted, JIGSAWS, surgery, surgical, gesture, fine-grained, surgeme, action, trajectory, segmentation, recognition, parsing. Selected articles were classified based on the level of supervision required for training and divided into different groups representing major frameworks for time series analysis and data modelling. Results: A total of 52 articles were reviewed. The research field is showing rapid expansion, with the majority of articles published in the last 4 years. Deep-learning-based temporal models with discriminative feature extraction and multi-modal data integration have demonstrated promising results on small surgical datasets. Currently, unsupervised methods perform significantly less well than the supervised approaches. Conclusion: The development of large and diverse open-source datasets of annotated demonstrations is essential for development and validation of robust solutions for surgical gesture recognition. While new strategies for discriminative feature extraction and knowledge transfer, or unsupervised and semi-supervised approaches, can mitigate the need for data and labels, they have not yet been demonstrated to achieve comparable performance. Important future research directions include detection and forecast of gesture-specific errors and anomalies. Significance: This paper is a comprehensive and structured analysis of surgical gesture recognition methods aiming to summarize the status of this rapidly evolving field.

show abstract

Section: Translational Researchmentioning

confidence: 99%

Gesture Recognition in Robotic Surgery: A Review

Amsterdam

Clarkson²,

Stoyanov³

2021

IEEE Trans. Biomed. Eng.

View full text Add to dashboard Cite

show abstract

“…Due to the limited experimental conditions and the lack of real surgical materials in hospitals, this paper uses the Lap Game endoscopic surgery simulator as the data acquisition platform and selects several commonly used endoscopic surgical instruments. On this basis, with reference to the sampling and annotation methods of the relevant literature (Bawa et al, 2021(Bawa et al, , 2020, the relevant simulation surgery videos were recorded, and the images of different video frames were extracted at equal intervals through Python script files, which were used to evenly sample the image information of each stage of the surgery process, so as to obtain the endoscopic simulation surgery process data set. The relevant information is shown in Table 1.…”

Section: Data Set Creationmentioning

confidence: 99%

Development of a flexible endoscopic robot with autonomous tracking control ability using machine vision and deep learning

Qian,

Zhang,

Pei

et al. 2024

Mech. Sci.

View full text Add to dashboard Cite

Abstract. A flexible endoscopic robot is designed to solve the problem that it is difficult for auxiliary doctors to maintain a stable visual field in traditional endoscopic surgery. Based on geometric derivation, a motion control method under the constraint of the remote center motion (RCM) of the robot system is established, and a set of circular trajectories are planned for it. The RCM error of the robot during operation and the actual trajectory of the robot end in three-dimensional space are obtained through the motion capture system. The end of the robot is controlled by the heterogeneous primary–secondary teleoperation control algorithm based on position increments. Finally, the RTMDet deep learning object detection algorithm was selected to identify and locate surgical instruments through comparative experiments, and the autonomous tracking control was completed based on visual guidance. In the process of autonomous tracking, the RCM error was less than 1 mm, which met the actual surgical requirements.

show abstract

ESAD: Endoscopic Surgeon Action Detection Dataset

Cited by 2 publications

References 0 publications

Gesture Recognition in Robotic Surgery: A Review

Gesture Recognition in Robotic Surgery: A Review

Development of a flexible endoscopic robot with autonomous tracking control ability using machine vision and deep learning

Contact Info

Product

Resources

About