Robot Interaction Control in Medicine and Surgery: Original Results and Open Problems

Bayle, B.; Joinie-Maurin, M.; Barbé, Laurent; Gangloff, Jacques; Mathelin, Michel de

doi:10.1007/978-1-4614-8648-0_11

Cited by 12 publications

(14 citation statements)

References 26 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(3) Dropout applied to layer 1 (L1). (4) Dropout applied to layer 2 (L2). was stopped after observing that the loss value reached a plateau, and there was no visible improvement in test set accuracy.…”

Section: Lstm Network Optimizationmentioning

confidence: 99%

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

Marbán

Srinivasan

Samek

et al. 2019

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

Providing force feedback as relevant information in current Robot-Assisted Minimally Invasive Surgery systems constitutes a technological challenge due to the constraints imposed by the surgical environment. In this context, Sensorless Force Estimation techniques represent a potential solution, enabling to sense the interaction forces between the surgical instruments and soft-tissues. Specifically, if visual feedback is available for observing soft-tissues' deformation, this feedback can be used to estimate the forces applied to these tissues. To this end, a force estimation model, based on Convolutional Neural Networks and Long-Short Term Memory networks, is proposed in this work. This model is designed to process both, the spatiotemporal information present in video sequences and the temporal structure of tool data (the surgical tool-tip trajectory and its grasping status). A series of analyses are carried out to reveal the advantages of the proposal and the challenges that remain for real applications. This research work focuses on two surgical task scenarios, referred to as pushing and pulling tissue. For these two scenarios, different input data modalities and their effect on the force estimation quality are investigated. These input data modalities are tool data, video sequences and a combination of both. The results suggest that the force estimation quality is better when both, the tool data and video sequences, are processed by the neural network model. Moreover, this study reveals the need for a loss function, designed to promote the modeling of smooth and sharp details found in force signals. Finally, the results show that the modeling of forces due to pulling tasks is more challenging than for the simplest pushing actions.

show abstract

Section: Lstm Network Optimizationmentioning

confidence: 99%

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

Marbán

Srinivasan

Samek

et al. 2019

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

show abstract

“…Given M v samples from a dataset (i.e. images), the discriminator network D φ updates its parameters φ with the loss function defined in (1). A reconstructed imageX is computed by the generator network G θ as an encoding-decoding process.…”

Section: Model Optimizationmentioning

confidence: 99%

“…For instance, improved manipulation of human soft-tissues avoiding damage due to excessive applied forces. However, the integration of force feedback in surgical robotic systems still remains an open problem [1] [2]. Advances in the fields of computer vision and artificial intelligence have resulted in an emerging research area known as Vision Based Force Sensing (VBFS).…”

Section: Introductionmentioning

confidence: 99%

Estimation of Interaction Forces in Robotic Surgery using a Semi-Supervised Deep Neural Network Model

Srinivasan

Samek

Fernandez

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Providing force feedback as a feature in current Robot-Assisted Minimally Invasive Surgery systems still remains a challenge. In recent years, Vision-Based Force Sensing (VBFS) has emerged as a promising approach to address this problem. Existing methods have been developed in a Supervised Learning (SL) setting. Nonetheless, most of the video sequences related to robotic surgery are not provided with ground-truth force data, which can be easily acquired in a controlled environment. A powerful approach to process unlabeled video sequences and find a compact representation for each video frame relies on using an Unsupervised Learning (UL) method. Afterward, a model trained in an SL setting can take advantage of the available ground-truth force data. In the present work, UL and SL techniques are used to investigate a model in a Semi-Supervised Learning (SSL) framework, consisting of an encoder network and a Long-Short Term Memory (LSTM) network. First, a Convolutional Auto-Encoder (CAE) is trained to learn a compact representation for each RGB frame in a video sequence. To facilitate the reconstruction of high and low frequencies found in images, this CAE is optimized using an adversarial framework and a L1-loss, respectively. Thereafter, the encoder network of the CAE is serially connected with an LSTM network and trained jointly to minimize the difference between ground-truth and estimated force data. Datasets addressing the force estimation task are scarce. Therefore, the experiments have been validated in a custom dataset. The results suggest that the proposed approach is promising.

show abstract

“…Despite all benefits, one of the major limitations in the current robotic surgical systems is the lack of force feedback, which is considered an open problem. 1 Force feedback is of utmost importance to prevent tissue damage, and increase both surgeon-patient transparency and procedure accuracy. Moreover, it is associated with task completion time and increased intraoperative injury.…”

Section: Introductionmentioning

confidence: 99%

Exploring the effects of dimensionality reduction in deep networks for force estimation in robotic-assisted surgery

et al. 2016

View full text Add to dashboard Cite

Robotic-Assisted Surgery approach overcomes the limitations of the traditional laparoscopic and open surgeries. However, one of its major limitations is the lack of force feedback. Since there is no direct interaction between the surgeon and the tissue, there is no way of knowing how much force the surgeon is applying which can result in irreversible injuries. The use of force sensors is not practical since they impose different constraints. Thus, we make use of a neuro-visual approach to estimate the applied forces, in which the 3D shape recovery together with the geometry of motion are used as input to a deep network based on LSTM-RNN architecture. When deep networks are used in real time, pre-processing of data is a key factor to reduce complexity and improve the network performance. A common pre-processing step is dimensionality reduction which attempts to eliminate redundant and insignificant information by selecting a subset of relevant features to use in model construction. In this work, we show the effects of dimensionality reduction in a real-time application: estimating the applied force in Robotic-Assisted Surgeries. According to the results, we demonstrated positive effects of doing dimensionality reduction on deep networks including: faster training, improved network performance, and overfitting prevention. We also show a significant accuracy improvement, ranging from about 33% to 86%, over existing approaches related to force estimation.

show abstract

Robot Interaction Control in Medicine and Surgery: Original Results and Open Problems

Cited by 12 publications

References 26 publications

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

Estimation of Interaction Forces in Robotic Surgery using a Semi-Supervised Deep Neural Network Model

Exploring the effects of dimensionality reduction in deep networks for force estimation in robotic-assisted surgery

Contact Info

Product

Resources

About