Machine learning-based augmented reality for improved surgical scene understanding

Pauly, Olivier; Diotte, Benoît; Fallavollita, Pascal; Weidert, Simon; Euler, Ekkehard; Navab, Nassir

doi:10.1016/j.compmedimag.2014.06.007

Cited by 39 publications

(23 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Another approach to enhance surgical quality and efficacy is to assist surgeons by augmenting reality and projecting X-ray structures and spatial information regarding current surgical instrument positions on the patient’s body. A feasibility study by Pauly et al [58] showed that larger instruments could be confidently located from depth data, but small and reflective objects, including scalpels, could not be accurately segmented. To overcome this, a marker-based instrument tracking system using two Kinects was suggested to help surgical trainees to learn computer-assisted surgery [59].…”

Section: Kinect Imaging For Healthcarementioning

confidence: 99%

Evaluation of Kinect 3D Sensor for Healthcare Imaging

et al. 2016

View full text Add to dashboard Cite

Microsoft Kinect is a three-dimensional (3D) sensor originally designed for gaming that has received growing interest as a cost-effective and safe device for healthcare imaging. Recent applications of Kinect in health monitoring, screening, rehabilitation, assistance systems, and intervention support are reviewed here. The suitability of available technologies for healthcare imaging applications is assessed. The performance of Kinect I, based on structured light technology, is compared with that of the more recent Kinect II, which uses time-of-flight measurement, under conditions relevant to healthcare applications. The accuracy, precision, and resolution of 3D images generated with Kinect I and Kinect II are evaluated using flat cardboard models representing different skin colors (pale, medium, and dark) at distances ranging from 0.5 to 1.2 m and measurement angles of up to 75°. Both sensors demonstrated high accuracy (majority of measurements <2 mm) and precision (mean point to plane error <2 mm) at an average resolution of at least 390 points per cm2. Kinect I is capable of imaging at shorter measurement distances, but Kinect II enables structures angled at over 60° to be evaluated. Kinect II showed significantly higher precision and Kinect I showed significantly higher resolution (both p < 0.001). The choice of object color can influence measurement range and precision. Although Kinect is not a medical imaging device, both sensor generations show performance adequate for a range of healthcare imaging applications. Kinect I is more appropriate for short-range imaging and Kinect II is more appropriate for imaging highly curved surfaces such as the face or breast.

show abstract

Section: Kinect Imaging For Healthcarementioning

confidence: 99%

Evaluation of Kinect 3D Sensor for Healthcare Imaging

et al. 2016

View full text Add to dashboard Cite

show abstract

“…video surveillance [1], augmented reality [2], or analysis of people behavior [3]) typically include strategies for separating the moving objects (MOs) in the scene, called foreground (FG), from the static information, called background (BG). These strategies, commonly known as background subtraction strategies, have been widely studied since the 90s and currently there are thousands of algorithms to carry them out.…”

Section: Introductionmentioning

confidence: 99%

Detection of stationary foreground objects: A survey

Cuevas

Martínez

García

2016

Computer Vision and Image Understanding

View full text Add to dashboard Cite

“…In order to finalize the embedding process, we compute the singular value decomposition of the graph feature matrix, and concatenate the first couple singular values and vectors (8). When using quadratic, antisymmetric (T ji = T ij T ) feature transform functions the graph feature matrix is guaranteed to be symmetric.…”

Section: Graph Node Embedding Frameworkmentioning

confidence: 99%

“…While most methods use 2D visual information only [2], there are numerous 3D shape based recognition techniques [3,4], as well as methods that use both visual and shape information [5,6]. Object detection methods are essential for scene understanding [7], which has a number of applications in different fields, such as robotics [1] or augmented reality [8].…”

Section: Introductionmentioning

confidence: 99%

3D Object Detection and Scene Optimization for Tangible Augmented Reality

Szemenyei

Vajda

2018

Period. Polytech. Elec. Eng. Comp. Sci.

View full text Add to dashboard Cite

Object recognition in 3D scenes is one of the fundamental tasks in computer vision. It is used frequently in robotics or augmented reality applications [1]. In our work we intend to apply 3D shape recognition to create a Tangible Augmented Reality system that is able to pair virtual and real objects in natural indoors scenes. In this paper we present a method for arranging virtual objects in a real-world scene based on primitive shape graphs. For our scheme, we propose a graph node embedding algorithm for graphs with vectorial nodes and edges, and genetic operators designed to improve the quality of the global setup of virtual objects. We show that our methods improve the quality of the arrangement significantly.

show abstract

Machine learning-based augmented reality for improved surgical scene understanding

Cited by 39 publications

References 1 publication

Evaluation of Kinect 3D Sensor for Healthcare Imaging

Evaluation of Kinect 3D Sensor for Healthcare Imaging

Detection of stationary foreground objects: A survey

3D Object Detection and Scene Optimization for Tangible Augmented Reality

Contact Info

Product

Resources

About