TSC-DL: Unsupervised trajectory segmentation of multi-modal surgical demonstrations with Deep Learning

Murali, Adithyavairavan; Garg, Animesh; Krishnan, Sanjay; Pokorny, Florian T.; Abbeel, Pieter; Darrell, Trevor; Goldberg, Ken

doi:10.1109/icra.2016.7487607

Cited by 58 publications

(45 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The range of NMI is [0,1], where 0 means that there is no correlation between two clustering results, while 1 represents the results are completely related. We compare the proposed method TSC-SCAE with stateof-the-art methods, including TSC [8], GMM [7], TSC-VGG, TSC-SIFT [9] and TSC-SCAE on the selected surgical demonstrations. According to the data source in the different methods, the experiments are divided into two categories: one use kinematics data alone and the other use both video and kinematic data.…”

Section: Resultsmentioning

confidence: 99%

“…Recently, video data are involved by using a deep learning based method, since traditional pattern recognition based feature extraction methods can't model the variations among surgeon's videos well. A. Murali et al [9] employ VGGNet to extract features from video followed by Transition State Clustering (TSC) for task-level segmentation using both kinematic and video data. Although the involvement of video source enables the higher accuracy of segmentation, the feature extraction from videos is timeconsuming and easily leads to over-segmentation.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Trajectory Segmentation and Promoting of Multi-Modal Surgical Demonstrations

Shao

Zhao

Xie

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

To improve the efficiency of surgical trajectory segmentation for robot learning in robot-assisted minimally invasive surgery, this paper presents a fast unsupervised method using video and kinematic data, followed by a promoting procedure to address the over-segmentation issue. Unsupervised deep learning network, stacking convolutional auto-encoder, is employed to extract more discriminative features from videos in an effective way. To further improve the accuracy of segmentation, on one hand, wavelet transform is used to filter out the noises existed in the features from video and kinematic data. On the other hand, the segmentation result is promoted by identifying the adjacent segments with no state transition based on the predefined similarity measurements. Extensive experiments on a public dataset JIGSAWS show that our method achieves much higher accuracy of segmentation than state-of-the-art methods in the shorter time.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Unsupervised Trajectory Segmentation and Promoting of Multi-Modal Surgical Demonstrations

Shao

Zhao

Xie

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

show abstract

“…77 Deep learning techniques have also been used for segmentation, where pre-trained architectures were trained on non-surgical image libraries. 78 Segmentation of surgical tasks is still an open problem, as it is concerned with the difficult problem of assigning labels to highly variable time-series data. A surgical activity dataset by Johns Hopkins University and Intuitive Surgical Inc. consisting of motion and video data is available for researchers interested in this problem.…”

Section: Learning From Humansmentioning

confidence: 99%

Robot Autonomy for Surgery

Yip¹,

Das²

2018

The Encyclopedia of Medical Robotics

View full text Add to dashboard Cite

Autonomous surgery involves having surgical tasks performed by a robot operating under its own will, with partial or no human involvement. There are several important advantages of automation in surgery, which include increasing precision of care due to sub-millimeter robot control, real-time utilization of biosignals for interventional care, improvements to surgical efficiency and execution, and computer-aided guidance under various medical imaging and sensing modalities. While these methods may displace some tasks of surgical teams and individual surgeons, they also present new capabilities in interventions that are too difficult or go beyond the skills of a human. In this chapter, we provide an overview of robot autonomy in commercial use and in research, and present some of the challenges faced in developing autonomous surgical robots.

show abstract

“…Existing surgical robotics platforms rely on pure teleoperation through a master-slave interface where the surgeon fully controls the motions of the robot. To reduce tedium and fatigue in long or repetitive procedures, recent work has highlighted several opportunities for autonomous execution of surgical subtasks [36] including debridement [15], suturing [24,[31][32][33], and palpation for tumor detection [6].…”

Section: Introductionmentioning

confidence: 99%

Fast and Reliable Autonomous Surgical Debridement with Cable-Driven Robots Using a Two-Phase Calibration Procedure

Seita¹,

Krishnan²,

Fox³

et al. 2018

2018 IEEE International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

Automating precision subtasks such as debridement (removing dead or diseased tissue fragments) with Robotic Surgical Assistants (RSAs) such as the da Vinci Research Kit (dVRK) is challenging due to inherent non-linearities in cabledriven systems. We propose and evaluate a novel two-phase coarse-to-fine calibration method. In Phase I (coarse), we place a red calibration marker on the end effector and let it randomly move through a set of open-loop trajectories to obtain a large sample set of camera pixels and internal robot end-effector configurations. This coarse data is then used to train a Deep Neural Network (DNN) to learn the coarse transformation bias. In Phase II (fine), the bias from Phase I is applied to move the end-effector toward a small set of specific target points on a printed sheet. For each target, a human operator manually adjusts the end-effector position by direct contact (not through teleoperation) and the residual compensation bias is recorded. This fine data is then used to train a Random Forest (RF) to learn the fine transformation bias. Subsequent experiments suggest that without calibration, position errors average 4.55mm. Phase I can reduce average error to 2.14mm and the combination of Phase I and Phase II can reduces average error to 1.08mm. We apply these results to debridement of raisins and pumpkin seeds as fragment phantoms. Using an endoscopic stereo camera with standard edge detection, experiments with 120 trials achieved average success rates of 94.5%, exceeding prior results with much larger fragments (89.4%) and achieving a speedup of 2.1x, decreasing time per fragment from 15.8 seconds to 7.3 seconds. Source code, data, and videos are available at https://sites.google.com/ view/calib-icra/.

show abstract

TSC-DL: Unsupervised trajectory segmentation of multi-modal surgical demonstrations with Deep Learning

Cited by 58 publications

References 18 publications

Unsupervised Trajectory Segmentation and Promoting of Multi-Modal Surgical Demonstrations

Unsupervised Trajectory Segmentation and Promoting of Multi-Modal Surgical Demonstrations

Robot Autonomy for Surgery

Fast and Reliable Autonomous Surgical Debridement with Cable-Driven Robots Using a Two-Phase Calibration Procedure

Contact Info

Product

Resources

About