Sight to touch: 3D diffeomorphic deformation recovery with mixture components for perceiving forces in robotic-assisted surgery

Abstract

Robotic-assisted minimally invasive surgical systems suffer from one major limitation which is the lack of interaction forces feedback. The restricted sense of touch hinders the surgeons' performance and reduces their dexterity and precision during a procedure. In this work, we present a sensory substitution approach that relies on visual stimuli to transmit the tool-tissue interaction forces to the operating surgeon. Our approach combines a 3D diffeomorphic deformation mapping with a generative model to precisely label the force level. The main highlights of our approach are that the use of diffeomorphic transformation ensures anatomical structure preservation and the label assignment is based on a parametric form of several mixture elements. We performed experimentations on both ex-vivo and in-vivo datasets and offer careful numerical results evaluating our approach. The results show that our solution has an error measure less than 1mm in all directions and an average labeling error of 2.05%. It can also be applicable to other scenarios that require force feedback such as microsurgery, knot tying or needle-based procedures.

Robótica en cirugía y neurocirugía, aplicaciones y desafíos, una revisión

“…que al variar cargas eléctricas puntuales en una pantalla capacitiva crean la sensación de percibir distintas texturas como superficie áspera, lisa, y pegajosa [36]; retroalimentación por actuadores de vibración localizados en la interfaz de control en contacto con los dedos del cirujano [37]. Realimentación visual donde la presión encontrada por el sensor es representada en barra de colores según la fuerza aplicada [38]. Electroestimulación muscular en la cual electrodos sobre la piel realizan descargas variando la frecuencia e intensidad de las señales eléctricas, al respecto Duente y otros [39] realizaron un arreglo de 20 canales de electrodos ubicados en el antebrazo para la transmisión de señales hápticas, pero los resultados muestran que se requiere un correcto sistema de mapeo y calibración de las sensaciones percibidas.…”

Motion Planning for Cutting Flexible Objects Based on Contact State Recognition

“…For instance, deboning meat involves the separation of materials with different properties, such as flesh and bones (1) . In rehabilitation (2)(3)(4) and surgeries, parts of tissues, such as bones, tendons, and muscles, need to be deformed locally (5)(6)(7) based on the intention of caregivers and surgeons. To process only parts of tissues appropriately without damaging surrounding tissues, the elastic and plastic deformations of heterogeneous objects must be distinguished accurately.…”

Occlusion-robust scene flow-based tissue deformation recovery incorporating a mesh optimization model

“…Second, even though robotic minimally invasive surgery (RMIS) has been proven to achieve many positive clinical outcomes in many cases [4,5], the absence of tactile sensation is still one of the shortcomings, which may lead to unintentional tissue injury and complicates the manipulation [6]. One possible approach to restoring the tactile sensation is to establish a force feedback system based on the observation and analysis of tissue deformation [6,7].…”

“…Another deformation recovery method was proposed by Aviles et al, implementing diffeomorphic deformation mapping in an unsupervised learning approach. The method was demonstrated to be useful in both ex vivo and in vivo datasets [7]. However, only 36 pairs of surface features were directly tracked, which was not sufficient for surface strain analysis.…”