&lt;title&gt;Augmented-reality visualization of brain structures with stereo and kinetic depth cues: system description and initial evaluation with head phantom&lt;/title&gt;

Maurer, Calvin R.; Sauer, Frank; Hu, Bo; Bascle, B.; Geiger, Bernhard; Wenzel, Fabian; Recchi, Filippo; Rohlfing, Torsten; Brown, Christopher R.; Bakos, Robert; Maciunas, Robert J.; Bani‐Hashemi, A

doi:10.1117/12.428086

Cited by 32 publications

(30 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The development of faster computers and the increasing quality of small displays made possible the development of head mounted displays (HMD) equipped with AR. Video see through systems and some of their application are described in [13,17,9,16], where the term "in-situ" visualization can be found.…”

Section: Introductionmentioning

confidence: 99%

Calibration of an optical see-through head-mounted display with variable zoom and focus for applications in computer-assisted interventions

Figl

Birkfellner

Hummel

et al. 2003

Medical Imaging 2003: Visualization, Image-Guided Procedures, and Display

View full text Add to dashboard Cite

We are developing an optical see through head mounted display in which preoperative planning data provided by a computer aided surgery system is overlaid to the optical image of the patient. In order to cope with head movements of the surgeon the device has to be calibrated for a wide zoom and focus range. For such a calibration accurate and robust localization of a huge amount of calibration points is of utmost importance. Because of the negligible radial distortion of the optics in our device, we were able to use projective invariants for stable detection of the calibration fiducials on a planar grid. The pattern at the planar grid was designed using a different cross ratio for four consecutive points in x respectively y direction. For automated image processing we put a CCD camera behind the eye piece of the device. The resulting image was thresholded and segmented, after deleting the artefacts a Sobel edge detector was applied and the image was Hough transformed to detect the x and y axes. Then the world coordinates of fiducial points on the grid could be detected. A series of six camera calibrations with two zoom settings was done.The mean values of the errors for the two calibrations were 0.08 mm respectively 0.3 mm.

show abstract

Section: Introductionmentioning

confidence: 99%

Calibration of an optical see-through head-mounted display with variable zoom and focus for applications in computer-assisted interventions

Figl

Birkfellner

Hummel

et al. 2003

Medical Imaging 2003: Visualization, Image-Guided Procedures, and Display

View full text Add to dashboard Cite

show abstract

“…To account for this limitation, AR with 3D graphics has been proposed by [6], but only to display the 3D surface of the target lesion. In [7] a head-mounted display with stereo is used to visualize brain structures. The 3D information can also be overlayed onto a 2D video of the patient [8], [9], [10] displayed on an external screen, by registering the images to the patient and computing camera pose.…”

Section: Introductionmentioning

confidence: 99%

Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: methods and performance evaluation

Paul¹,

Fleig²,

Jannin³

2005

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Abstract-This paper presents a new method for displaying in the same 3D scene multimodal preoperative images of a patient and images of the operative field viewed through surgical microscope binoculars for image-guided neurosurgery.Matching real world information, i.e., the operative field, with virtual world information, i.e., preoperative images of the patient, is an important issue in image-guided neurosurgery. This can be achieved by superimposing preoperative images onto a surgical microscope ocular or a head-mounted display. Such an approach is usually called augmented reality (AR). When surgery is performed in functional areas, such as the eloquent cortex, multimodal images are required. Therefore, preoperative images consist of a complex 3D multimodal scene which can hamper the vision of the real world when displayed in the neurosurgeons view of the operative field.The approach, introduced in this paper, is called augmented virtuality (AV) and involves displaying the operative field view in the virtual world, i.e., in the 3D multimodal scene which includes preoperative images of the patient. Information from the operative field consists of a 3D surface reconstructed from two stereoscopic images from surgical microscope binoculars using stereovision methods. As the microscope is part of a neuronavigation system and is tracked by an optical 3D localizer, the 3D reconstructed surface is directly expressed in the physical space coordinate system. Using the image-to-physical space transformation computed by the neuronavigation system, this 3D surface can also be directly expressed in the image coordinate system.In this paper, we present the method for reconstructing 3D surfaces of the operative field from stereoscopic views and matching the reconstructed surface with the preoperative images. Performance evaluation of this method was performed using a physical skull phantom. 300 image pairs of this phantom were acquired. The distance between the reconstructed surfaces and the skull surface segmented from a CT data set of this phantom was used as a system accuracy measurement. Our method was used for 6 clinical cases with lesions in eloquent areas. For the minimum microscope focus value, 3D reconstruction accuracy alone was shown to be within 1mm (median: 0.76mm ± 0.27), whereas virtual and real image matching accuracy was shown to be within 3mm (median: 2.29mm ± 0.59), including the imageto-physical space registration error.Clinical use of this system has proved the relevance of our approach. In addition to seeing beyond the surface, augmented virtuality can be used to see around the surgical area. With this system, neurosurgeons and clinical staff in the OR were able to interact with the resulting 3D scene by rotating and modifying transparency features. This AV system facilitates understanding of the spatial relationship between the operative field and the complex 3D multimodal scene, which includes preoperative images of the patient. Index Terms-Preoperative and Intraoperative Multimodal

show abstract

“…The user's spatial perception is based on stereo depth cues, and also on the kinetic depth cues that he receives with the viewpoint variations. Our system has been put into a neurosurgical context [11], adapted to an interventional MRI operating room [12,13], and has also been integrated with an ultrasound scanner [14,15].…”

Section: Introductionmentioning

confidence: 99%

An Augmented Reality Navigation System with a Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial

Sauer

Khamene

Vogt

2002

Medical Image Computing and Computer-Assisted Intervention — MICCAI 2002

View full text Add to dashboard Cite

Abstract. We extended a system for augmented reality visualization to include the capability for instrument tracking. The original system is based on a videosee-through head-mounted display and features single-camera tracking. The tracking camera is head-mounted, rigidly fixed to a stereo pair of cameras that provide a live video view of a workspace. The tracker camera includes an infrared illuminator and works in conjunction with a set of retroreflective markers that are placed around the workspace. This marker frame configuration delivers excellent pose information for a stable overlay of graphics onto the video images. Using the single camera also for instrument tracking with relatively small marker clusters, however, encounters problems of marker identification and of noise in the pose data. We present a multilevel planar marker design, which we used to build a needle placement phantom. In this phantom, we achieved a stable augmentation; the user can see the location of the hidden target and the needle without perceptible jitter of the overlaid graphics. Piercing the needle through a foam window and hitting the target is then intuitive and comfortable.Over a hundred users have tested the system, and are consistently able to correctly place the needle on the 6mm target without prior training.

show abstract

<title>Augmented-reality visualization of brain structures with stereo and kinetic depth cues: system description and initial evaluation with head phantom</title>

Cited by 32 publications

References 28 publications

Calibration of an optical see-through head-mounted display with variable zoom and focus for applications in computer-assisted interventions

Calibration of an optical see-through head-mounted display with variable zoom and focus for applications in computer-assisted interventions

Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: methods and performance evaluation

An Augmented Reality Navigation System with a Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial

Contact Info

Product

Resources

About