Direct Visual and Haptic Volume Rendering of Medical Data Sets for an Immersive Exploration in Virtual Reality

Faludi, Balázs; Zoller, Esther I.; Gerig, Nicolas; Zam, Azhar; Rauter, Georg; Cattin, Philippe C.

doi:10.1007/978-3-030-32254-0_4

Cited by 18 publications

(9 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Scholl et al (2018) do so in VR, although with quite small volumes (with largest side of 256 or 512 voxels). Faludi et al (2019) also use direct volume rendering and add haptics. In this case the largest volume side was 512 voxels.…”

Section: Immersive Visualizationmentioning

confidence: 99%

Inspection of Histological 3D Reconstructions in Virtual Reality

Lobachev¹,

Berthold²,

Pfeffer³

et al. 2021

Front. Virtual Real.

View full text Add to dashboard Cite

3D reconstruction is a challenging current topic in medical research. We perform 3D reconstructions from serial sections stained by immunohistological methods. This paper presents an immersive visualization solution to quality control (QC), inspect, and analyze such reconstructions. QC is essential to establish correct digital processing methodologies. Visual analytics, such as annotation placement, mesh painting, and classification utility, facilitates medical research insights. We propose a visualization in virtual reality (VR) for these purposes. In this manner, we advance the microanatomical research of human bone marrow and spleen. Both 3D reconstructions and original data are available in VR. Data inspection is streamlined by subtle implementation details and general immersion in VR.

show abstract

Section: Immersive Visualizationmentioning

confidence: 99%

Inspection of Histological 3D Reconstructions in Virtual Reality

Lobachev¹,

Berthold²,

Pfeffer³

et al. 2021

Front. Virtual Real.

View full text Add to dashboard Cite

show abstract

“…Recently, portable VR technologies, from simple cardboard inserts for smartphones to sophisticated VR headsets with accurate tracking sensors, became widely available (El Beheiry et al 2019). The advancement of VR technologies led to flourishing research to exploit portable VR to visualize volumetric data (Cohen et al 2013;Chan et al 2013;Faludi et al 2019), and the results have been promising because of the immersive environment. As VR improves user experience in volume exploration and visualization, we set up our proposed system entirely in VR.…”

Section: Related Workmentioning

confidence: 99%

“…Using VR to explore 3D volumes has big potentials (El Beheiry et al 2019;Cohen et al 2013;Chan et al 2013;Faludi et al 2019). Coupled with our touch-less NUI, VR will undoubtedly make NUI-VR 2 more innovative, efficient, and enjoyable because of its realistic and immersive nature.…”

Section: Virtual Realitymentioning

confidence: 99%

A VR-based volumetric medical image segmentation and visualization system with natural human interaction

Gao

Chang

et al. 2021

Virtual Reality

View full text Add to dashboard Cite

Volume rendering produces informative two-dimensional (2D) images from a 3-dimensional (3D) volume. It highlights the region of interest and facilitates a good comprehension of the entire data set. However, volume rendering faces a few challenges. First, a high-dimensional transfer function is usually required to differentiate the target from its neighboring objects with subtle variance. Unfortunately, designing such a transfer function is a strenuously trial-and-error process. Second, manipulating/visualizing a 3D volume with a traditional 2D input/output device suffers dimensional limitations. To address all the challenges, we design NUI-VR$$^2$$ 2 , a natural user interface-enabled volume rendering system in the virtual reality space. NUI-VR$$^2$$ 2 marries volume rendering and interactive image segmentation. It transforms the original volume into a probability map with image segmentation. A simple linear transfer function will highlight the target well in the probability map. More importantly, we set the entire image segmentation and volume rendering pipeline in an immersive virtual reality environment with a natural user interface. NUI-VR$$^2$$ 2 eliminates the dimensional limitations in manipulating and perceiving 3D volumes and dramatically improves the user experience.

show abstract

“…Computing the haptic forces only for one interaction point is sufficient to palpate the surface of a structure [6]. However, this is not enough for our current application, because there can easily be multiple contact points between the shank of the pedicle probe and the surrounding pedicular bone.…”

Section: Haptic Renderingmentioning

confidence: 99%

“…In previous work, we presented a method for direct visual and haptic volume rendering of medical data sets in virtual reality (VR) [6]. Unlike previous approaches [2,13,21], our method allows rendering the patient-specific anatomy without relying on any mesh or surface generation, guarantees a passive, continuous force field, and showed stable haptic feedback even for stiff objects such as bony structures.…”

Section: Introductionmentioning

confidence: 99%

Force quantification and simulation of pedicle screw tract palpation using direct visuo-haptic volume rendering

et al. 2020

Self Cite

View full text Add to dashboard Cite

Purpose We present a feasibility study for the visuo-haptic simulation of pedicle screw tract palpation in virtual reality, using an approach that requires no manual processing or segmentation of the volumetric medical data set. Methods In a first experiment, we quantified the forces and torques present during the palpation of a pedicle screw tract in a real boar vertebra. We equipped a ball-tipped pedicle probe with a 6-axis force/torque sensor and a motion capture marker cluster. We simultaneously recorded the pose of the probe relative to the vertebra and measured the generated forces and torques during palpation. This allowed us replaying the recorded palpation movements in our simulator and to fine-tune the haptic rendering to approximate the measured forces and torques. In a second experiment, we asked two neurosurgeons to palpate a virtual version of the same vertebra in our simulator, while we logged the forces and torques sent to the haptic device. Results In the experiments with the real vertebra, the maximum measured force along the longitudinal axis of the probe was 7.78 N and the maximum measured bending torque was 0.13 Nm. In an offline simulation of the motion of the pedicle probe recorded during the palpation of a real pedicle screw tract, our approach generated forces and torques that were similar in magnitude and progression to the measured ones. When surgeons tested our simulator, the distributions of the computed forces and torques were similar to the measured ones; however, higher forces and torques occurred more frequently. Conclusions We demonstrated the suitability of direct visual and haptic volume rendering to simulate a specific surgical procedure. Our approach of fine-tuning the simulation by measuring the forces and torques that are prevalent while palpating a real vertebra produced promising results.

show abstract

Direct Visual and Haptic Volume Rendering of Medical Data Sets for an Immersive Exploration in Virtual Reality

Cited by 18 publications

References 14 publications

Inspection of Histological 3D Reconstructions in Virtual Reality

Inspection of Histological 3D Reconstructions in Virtual Reality

A VR-based volumetric medical image segmentation and visualization system with natural human interaction

Force quantification and simulation of pedicle screw tract palpation using direct visuo-haptic volume rendering

Contact Info

Product

Resources

About