MixIn3D: 3D Mixed Reality with ToF-Camera

Koch, Reinhard; Schiller, Ingo; Bartczak, Bogumil; Kellner, F.; Köser, Kevin

doi:10.1007/978-3-642-03778-8_10

Cited by 26 publications

(22 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…for touch-free navigation for 3D medical visualization (Soutschek et al, 2008) as well as image segmentation , e.g. used for AR applications (Koch et al, 2009). In the context of biomedical applications ToF cameras have been used as an imaging modality for respiratory motion gating (Schaller et al, 2008) and patient positioning (Placht et al, 2012;Schaller et al, 2009) in radiotherapy as well as for building patient-specific respiratory motion models Wentz et al, 2012).…”

Section: State-of-the-artmentioning

confidence: 99%

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

Maier-Hein

Mountney

Bartoli

et al. 2013

Medical Image Analysis

242

130

View full text Add to dashboard Cite

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-operative morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon's navigation capabilites by observing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted instruments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D optical imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions.

show abstract

Section: State-of-the-artmentioning

confidence: 99%

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

Maier-Hein

Mountney

Bartoli

et al. 2013

Medical Image Analysis

242

130

View full text Add to dashboard Cite

show abstract

“…It puts the least limitation on context and illumination conditions compared with any previous approach; 2) Robust: Due to the depth-buffering employed, this method can work regardless of the spatial relationship among involved virtual and real objects; 3) Fast: The authors take advantage of OpenGL texture and OpenGL Shading Language (GLSL) fragment shader to parallelize the sampling of depth map and rendering into the frame buffer. A recent publication (Koch, et al, 2009) describes a paralleled research effort that adopted a similar approach for TV production in indoor environments with 3D model constructed beforehand.…”

Section: Introductionmentioning

confidence: 99%

Occlusion Handling Method for Ubiquitous Augmented Reality Using Reality Capture Technology and GLSL

Dong¹,

Feng²,

Kamat³

2011

Computing in Civil Engineering (2011)

View full text Add to dashboard Cite

ABSTRACT:The primary challenge in generating convincing Augmented Reality (AR) graphics is to project 3D models onto a user's view of the real world and create a temporal and spatial sustained illusion that the virtual and real objects co-exist. Regardless of the spatial relationship between the real and virtual objects, traditional AR graphical engines break the illusion of co-existence by displaying the real world merely as a background, and superimposing virtual objects on the foreground. This research proposes a robust depth sensing and frame buffer algorithm for handling occlusion problems in ubiquitous AR applications. A high-accuracy Time-of-flight (TOF) camera is used to capture the depth map of the real-world in real time. The distance information is processed using the OpenGL Shading Language (GLSL) and rendered into the graphics depth buffer, allowing accurate depth resolution and hidden surface removal in composite AR scenes. The designed algorithm is validated in several indoor and outdoor experiments using the SMART AR framework.

show abstract

“…Algumas aplicações das imagens TOF já foram realizadas na prevenção de acidentes entre automóveis e segurança de pedestres (RINGBECK, 2007), Visão Computacional (WIEDEMANN et al, 2008), mapeamentos tridimensionais de superfícies em ambientes fechados para a movimentação robótica (MAY et al, 2008), Computação Gráfica (KOLB et al, 2009;KOCH et al, 2009), e análise de deformações (JAMTSHO, 2010).…”

Section: Introductionunclassified

Modelagem tridimensional de superfícies planas a partir de imagem TOF

Oliveira

Centeno

2011

Bol. Ciênc. Geod.

View full text Add to dashboard Cite

RESUMONeste artigo é apresentado um estudo para a modelagem tridimensional de superfícies utilizando imagens obtidas com uma câmara TOF. O sistema utilizado é a câmara PMD CamCube 2.0, que calcula a distância de objetos utilizando o princípio TOF (Time-Of-Flight). Inicialmente, a câmara foi calibrada, o que possibilitou a correção geométrica da imagem a partir de Parâmetros de Orientação Interior (POI). Outro fator que afeta a qualidade dos dados é a presença de ruído em função da iluminação exterior ao sistema, que pode ser controlada variando o tempo de integração na captação da imagem. Posteriormente realizou-se a segmentação da imagem TOF para identificar as superfícies planas do objeto estudado. A solução encontrada utiliza o cálculo dos autovalores locais em superfícies, em pequenas janelas móveis. Com isto, gera-se uma nova imagem com bordas evidenciadas. A nova imagem é segmentada através do algoritmo de deslocamento pela média, com o objetivo de obter superfícies planas do objeto de interesse. Os píxeis da imagem segmentada são então projetados ortogonalmente, gerando uma nuvem de pontos com coordenadas tridimensionais das superfícies. Finalmente, as dimensões do objeto são comparadas com medidas de referência, sendo verificado que a diferença na medida tridimensional obtida é de aproximadamente 2 centímetros.

show abstract

MixIn3D: 3D Mixed Reality with ToF-Camera

Cited by 26 publications

References 18 publications

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

Occlusion Handling Method for Ubiquitous Augmented Reality Using Reality Capture Technology and GLSL

Modelagem tridimensional de superfícies planas a partir de imagem TOF

Contact Info

Product

Resources

About