Interacting with 3D objects in a virtual environment using an intuitive gesture system

Manders, Corey; Farbiz, Farzam; Tang, Ka Yin; Miaolong, Yuan; Chong, Bryan; Guan, Chua Gim

doi:10.1145/1477862.1477869

Cited by 9 publications

(5 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Juckette et al [62] mention the use of Leap Motion device [76] to facilitate the gesture-interface for the VR experience exploring an ancient Maya temple [62]. Manders et al [77] proposed their technique that leverages the fact that HMDs leave the user's chin visible for tracking. Coupled with stereo camera data to capture the user's gestural input, it can be used to manipulate 3D objects in virtual space.…”

Section: Gestural Input As Means Of Photogrammetric Models Manipulati...mentioning

confidence: 99%

PhotoTwinVR: An Immersive System for Manipulation, Inspection and Dimension Measurements of the 3D Photogrammetric Models of Real-Life Structures in Virtual Reality

Tadeja,

Rydlewicz,

et al. 2019

Preprint

View full text Add to dashboard Cite

Photogrammetry is a science dealing with obtaining reliable information about physical objects using their imagery description. Recent advancements in the development of Virtual Reality (VR) can help to unlock the full potential offered by the digital 3D-reality models generated using the state-of-art photogrammetric technologies. These models are becoming a viable alternative for providing high-quality content for such immersive environment. Simultaneously, their analyses in VR could bring added-value to professionals working in various engineering and non-engineering settings and help in extracting useful information about physical objects. However, there is little research published to date on feasible interaction methods in the VR-based systems augmented with the 3D photogrammetric models, especially concerning gestural input interfaces. Consequently, this paper presents the PhotoTwinVR -an immersive, gesture-controlled system for manipulation and inspection of 3D photogrammetric models of physical objects in VR.Our system allows the user to perform basic engineering operations on the model subjected to the off-line inspection process. An observational study with a group of three domain-expert participants was completed to verify its feasibility. The system was populated with a 3D photogrammetric model of an existing pipe-rack generated using a commercial software package. The participants were asked to carry out a survey measurement of the object using the measurement toolbox offered by PhotoTwinVR. The study revealed a potential of such immersive tool to be applied in practical real-words cases of off-line inspections of pipelines.

show abstract

Section: Gestural Input As Means Of Photogrammetric Models Manipulati...mentioning

confidence: 99%

PhotoTwinVR: An Immersive System for Manipulation, Inspection and Dimension Measurements of the 3D Photogrammetric Models of Real-Life Structures in Virtual Reality

Tadeja,

Rydlewicz,

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…However, there is little prior work focusing on interacting with the generated 3D models in VR using virtual hands and unencumbered fully tracked gestural input driven by hand tracking. Prior work has investigated non-photogrammetric object manipulation in VR using mid-air interaction [7,16] and bimanual input techniques [24]. However, despite this area being under-explored, the literature does indicate potential for unencumbered hand tracking-driven interaction of photogrammetric 3D models.…”

Section: Introductionmentioning

confidence: 99%

Exploring gestural input for engineering surveys of real-life structures in virtual reality using photogrammetric 3D models

Tadeja

Rydlewicz³

et al. 2021

Multimed Tools Appl

View full text Add to dashboard Cite

Photogrammetry is a promising set of methods for generating photorealistic 3D models of physical objects and structures. Such methods may rely solely on camera-captured photographs or include additional sensor data. Digital twins are digital replicas of physical objects and structures. Photogrammetry is an opportune approach for generating 3D models for the purpose of preparing digital twins. At a sufficiently high level of quality, digital twins provide effective archival representations of physical objects and structures and become effective substitutes for engineering inspections and surveying. While photogrammetric techniques are well-established, insights about effective methods for interacting with such models in virtual reality remain underexplored. We report the results of a qualitative engineering case study in which we asked six domain experts to carry out engineering measurement tasks in an immersive environment using bimanual gestural input coupled with gaze-tracking. The qualitative case study revealed that gaze-supported bimanual interaction of photogrammetric 3D models is a promising modality for domain experts. It allows the experts to efficiently manipulate and measure elements of the 3D model. To better allow designers to support this modality, we report design implications distilled from the feedback from the domain experts.

show abstract

“…3D image technology plays an important role in virtual reality and augmented reality systems [1], [2]. Stereoscopic vision, as the simplest form of 3D visions, is widely used because it can provide a higher sense of reality than 2D images by just presenting a pair of stereo images corresponding to left and right eyesights.…”

Section: Introductionmentioning

confidence: 99%

Stereo Image Retargeting with Shift-Map

Nakashima

Utsugi

Takahashi

et al. 2011

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYWe propose a new stereo image retargeting method based on the framework of shift-map image editing. Retargeting is the process of changing the image size according to the target display while preserving as much of the richness of the image as possible, and is often applied to monocular images and videos. Retargeting stereo images poses a new challenge because pixel correspondences between the stereo pair should be preserved to keep the scene's structure. The main contribution of this paper is integrating a stereo correspondence constraint into the retargeting process. Among several retargeting methods, we adopt shift-map image editing because this framework can be extended naturally to stereo images, as we show in this paper. We confirmed the effectiveness of our method through experiments. key words: stereo image editing, image retargeting, shift-map image editing

show abstract

Interacting with 3D objects in a virtual environment using an intuitive gesture system

Cited by 9 publications

References 5 publications

PhotoTwinVR: An Immersive System for Manipulation, Inspection and Dimension Measurements of the 3D Photogrammetric Models of Real-Life Structures in Virtual Reality

PhotoTwinVR: An Immersive System for Manipulation, Inspection and Dimension Measurements of the 3D Photogrammetric Models of Real-Life Structures in Virtual Reality

Exploring gestural input for engineering surveys of real-life structures in virtual reality using photogrammetric 3D models

Stereo Image Retargeting with Shift-Map

Contact Info

Product

Resources

About