Unsupervised Trajectory Segmentation and Promoting of Multi-Modal Surgical Demonstrations

Shao, Zhenzhou; Zhao, Hongfa; Xie, Jiexin; Qu, Ying; Guan, Yong; Tan, Jindong

doi:10.1109/iros.2018.8593379

Cited by 7 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Luongo et al [37] achieve 71% mAP using only kinematic data as input to an RNN on the JIGSAWS dataset. Other works that include images as input data report results of 70.6% mAP [26] and 79.1% mAP [27]. Thus, we consider that we achieved a good result, especially taking into account that recognition is performed on tasks that do not follow a specific order in any of the attempts.…”

Section: Discussionmentioning

confidence: 80%

See 1 more Smart Citation

Instruments Detection and Descriptive Gesture Segmentation on a Robotic Surgical Maneuvers Dataset

Rivas-Blanco,

López-Casado,

Herrera-López

et al. 2024

Preprint

View full text Add to dashboard Cite

Large datasets play a crucial role in the progression of surgical robotics, facilitating advancements in the fields of surgical task recognition and automation. Moreover, public datasets enable the comparative analysis of various algorithms and methodologies, thereby assessing their effectiveness and performance. The ROSMA (Robotics Surgical Maneuvers) dataset provides 206 trials of common surgical training tasks performed with the da Vinci Research Kit (dVRK). In this work, we extend ROSMA dataset with two annotated subsets: ROSMAT24, which contains bounding box annotations for instrument detection, and ROSMAG40, which contains high and low level gesture annotations. We propose an annotation method that provides independent labels for the right-handed tool and the left-handed tool. For instrument identification, we validate our proposal with a YOLOv4 model in two experimental scenarios. We demonstrate the generalization capabilities of the network to detect instruments in unseen scenarios. For gesture segmentation, we propose two label categories: high-level annotations that describe gestures at a maneuvers level, and low-level annotations that describe gestures at a fine-grain level. To validate this proposal, we have designed a recurrent neural network based on a bidirectional long-short term memory layer. We present results for four cross-validation experimental setups, reaching up to a 77.35% mAP.

show abstract

Section: Discussionmentioning

confidence: 80%

“…It involves breaking trajectories into sub-trajectories, facilitating learning from demonstrations, skill assessment, phase recognition, among other applications. Authors leverage kinematics information provided by surgical robots, which when combined with video data, yields improved accuracy results [27] [28].…”

Section: Introductionmentioning

confidence: 99%

Instruments Detection and Descriptive Gesture Segmentation on a Robotic Surgical Maneuvers Dataset

Rivas-Blanco,

López-Casado,

Herrera-López

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…This work is based on a structure of dense connection, in which the first half of the network, the dense block, is an encoder that performs feature extraction, the transition layer performs the trajectory segmentation, and the up-sampling layer is used for image reconstruction. A similar approach is presented in [77], but using a compact stacking convolutional autoencoder model and wavelet transform based filtering. Marban et al [79] propose a method to estimate the position and velocity of the instruments in 3D from monocular videos using a regression model based on CNN+LSTM.…”

Section: ) Trajectory Segmentationmentioning

confidence: 99%

A Review on Deep Learning in Minimally Invasive Surgery

et al. 2021

View full text Add to dashboard Cite

In the last five years, deep learning has attracted great interest in computer-assisted systems for Minimally Invasive Surgery. The straightforward accessibility to images in surgical interventions makes deep neural networks enormously powerful for solving classification problems in complex surgical scenarios. The objective of this work is to provide readers a survey on deep learning models applied to minimally invasive surgery, identifying the different architectures used depending on the application, the results achieved until now, and the publicly available surgical datasets that can be used for validating new studies. A total of 85 publications have been extracted from manual research from four databases (IEEE Xplorer, Springer Link, Science Direct, and ACM Digital Library). After analyzing all these studies, they have been classified into four applications: surgical image analysis, surgical task analysis, surgical skill assessment, and automation of surgical tasks. This work provides a technical description of these works and a comparison among them. Finally, promising research directions to advance in this field are identified.

show abstract

“…It involves breaking trajectories into sub-trajectories, facilitating learning from demonstrations, skill assessment, phase recognition, among other applications. Authors take advantage of kinematics information provided by surgical robots, which combined with video data, allows more accurate results [26,27].…”

Section: Introductionmentioning

confidence: 99%

Instrument Detection and Descriptive Gesture Segmentation on a Robotic Surgical Maneuvers Dataset

Rivas-Blanco,

López-Casado,

Herrera-López

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Large datasets play a crucial role in the progression of surgical robotics, facilitating advancements in the fields of surgical task recognition and automation. Moreover, public datasets enable the comparative analysis of various algorithms and methodologies, thereby assessing their effectiveness and performance. The ROSMA (Robotics Surgical Maneuvers) dataset provides 206 trials of common surgical training tasks performed with the da Vinci Research Kit (dVRK). In this work, we extend the ROSMA dataset with two annotated subsets: ROSMAT24, which contains bounding box annotations for instrument detection, and ROSMAG40, which contains high and low-level gesture annotations. We propose an annotation method that provides independent labels for the right-handed tools and the left-handed tools. For instrument identification, we validate our proposal with a YOLOv4 model in two experimental scenarios. We demonstrate the generalization capabilities of the network to detect instruments in unseen scenarios. On the other hand, for gesture segmentation, we propose two label categories: high-level annotations that describe gestures at a maneuvers level, and low-level annotations that describe gestures at a fine-grain level. To validate this proposal, we have designed a recurrent neural network based on a bidirectional long-short term memory layer. We present results for four cross-validation experimental setups, reaching up to a 77.35% mAP.

show abstract

Unsupervised Trajectory Segmentation and Promoting of Multi-Modal Surgical Demonstrations

Cited by 7 publications

References 14 publications

Instruments Detection and Descriptive Gesture Segmentation on a Robotic Surgical Maneuvers Dataset

Instruments Detection and Descriptive Gesture Segmentation on a Robotic Surgical Maneuvers Dataset

A Review on Deep Learning in Minimally Invasive Surgery

Instrument Detection and Descriptive Gesture Segmentation on a Robotic Surgical Maneuvers Dataset

Contact Info

Product

Resources

About