Efficient Physics-Based Implementation for Realistic Hand-Object Interaction in Virtual Reality

Holl, M.; Oberweger, Markus; Arth, Clemens; Lepetit, Vincent

doi:10.1109/vr.2018.8448284

Cited by 70 publications

(46 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Subjects reported an experience of "inaccurate friction, " which caused objects to slip out of the virtual hand more often than they would have in RW. Unrealistic physics in object interactions in VR has been shown to have a negative impact on a user's experience (Lin et al, 2016;McMahan et al, 2016;Höll et al, 2018). This lack of accurate physics causes a mismatch between the user's perception of what should happen and what they are seeing.…”

Section: Discussion Virtual Reality and Real-world Task Completion Ratementioning

confidence: 99%

See 1 more Smart Citation

Comparison of Dexterous Task Performance in Virtual Reality and Real-World Environments

Joyner¹,

Vaughn-Cooke²,

Benz³

2021

Front. Virtual Real.

View full text Add to dashboard Cite

Virtual reality is being used to aid in prototyping of advanced limb prostheses with anthropomorphic behavior and user training. A virtual version of a prosthesis and testing environment can be programmed to mimic the appearance and interactions of its real-world counterpart, but little is understood about how task selection and object design impact user performance in virtual reality and how it translates to real-world performance. To bridge this knowledge gap, we performed a study in which able-bodied individuals manipulated a virtual prosthesis and later a real-world version to complete eight activities of daily living. We examined subjects' ability to complete the activities, how long it took to complete the tasks, and number of attempts to complete each task in the two environments. A notable result is that subjects were unable to complete tasks in virtual reality that involved manipulating small objects and objects flush with the table, but were able to complete those tasks in the real world. The results of this study suggest that standardization of virtual task environment design may lead to more accurate simulation of real-world performance.

show abstract

Section: Discussion Virtual Reality and Real-world Task Completion Ratementioning

confidence: 99%

“…This lack of accurate physics causes a mismatch between the user's perception of what should happen and what they are seeing. Improvements are being made to physics calculations to more accurately calculate how an object should respond to touch (Todorov et al, 2012;Höll et al, 2018).…”

Section: Discussion Virtual Reality and Real-world Task Completion Ratementioning

confidence: 99%

Comparison of Dexterous Task Performance in Virtual Reality and Real-World Environments

Joyner¹,

Vaughn-Cooke²,

Benz³

2021

Front. Virtual Real.

View full text Add to dashboard Cite

show abstract

“…These works are similar to our work on improving the grasp of objects. The work of Höll et al [22] presents the potential of efficient physics-based implementation for realistic handobject interaction in VR. Their work has a similar base to Oprea et al [23], which presents a visually plausible grasping system for object manipulation and interaction in VR, which is operated on different predefined objects.…”

Section: Grasping Methodsmentioning

confidence: 99%

The Effect of Semi-Transparent and Interpenetrable Hands on Object Manipulation in Virtual Reality

Veldhuizen

Yang

2021

IEEE Access

View full text Add to dashboard Cite

Manipulating objects with the hand is a common task in Virtual Reality (VR). However, some issues can occur during these manipulations. Occlusion is one issue that happens when the virtual hand covers an object, and therefore the perception of that object is hindered. Semi-transparent hands could solve this problem. Another issue is the difficulty of delicate hand object manipulation. By using interpenetrable hands, turning off the physics of the hands, this difficulty might decrease. Still, there is a lack of research into how significant the impact is while using these methods. In this paper, with the designs of semi-transparent hands and interpenetrable hands, we present the results of our conducted user study focusing on the effect of semi-transparent and interpenetrable hands on hand object manipulation tasks in virtual reality. The user study includes a VR environment where participants are asked to perform tasks. These tasks are recorded in objective results of accuracy and speed. Afterwards, they fill out a questionnaire about their opinion on the difficulty while using the different methods in which the subjective findings are recorded. Additionally, we improved the semi-transparent hands by adding a feature that smoothly transitions the hand from opaque to semi-transparent. As an input device, we used the Leap Motion Controller (LMC) for this user study. However, any hand tracking sensor that tracks hands from the VR headset's point of view could be used. Semi-transparent and interpenetrable hands have shown significant improvement for precise manipulation, which was verified by user feedback from the questionnaire and the data from the tasks.

show abstract

“…Regarding manipulation in a VR/AR, [17] and [18] used a bespoke hand glove and haptic sensors to capture hand information and mapped this information to either a realistic spring hand model or a simplistic two fingertips model. Physics-based approaches modelling hand meshes and forces for object interaction include [19,20,21]. [21] chose to use a spring hand model to infer contact forces, whereas [20] optimised hand deformation using a particle based approach.…”

Section: Related Workmentioning

confidence: 99%

Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitation

Antotsiou

Garcia-Hernando²,

Kim³

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Human hand actions are quite complex, especially when they involve object manipulation, mainly due to the high dimensionality of the hand and the vast action space that entails. Imitating those actions with dexterous hand models involves different important and challenging steps: acquiring human hand information, retargeting it to a hand model, and learning a policy from acquired data. In this work, we capture the hand information by using a state-of-the-art hand pose estimator. We tackle the retargeting problem from the hand pose to a 29 DoF hand model by combining inverse kinematics and PSO with a task objective optimisation. This objective encourages the virtual hand to accomplish the manipulation task, relieving the effect of the estimator's noise and the domain gap. Our approach leads to a better success rate in the grasping task compared to our inverse kinematics baseline, allowing us to record successful human demonstrations. Furthermore, we used these demonstrations to learn a policy network using generative adversarial imitation learning (GAIL) that is able to autonomously grasp an object in the virtual space. J 9 J 10 J J 19 J 16 J 20 J 13 J 14 J 15 max(finger y ) J 18 17

show abstract

Efficient Physics-Based Implementation for Realistic Hand-Object Interaction in Virtual Reality

Cited by 70 publications

References 46 publications

Comparison of Dexterous Task Performance in Virtual Reality and Real-World Environments

Comparison of Dexterous Task Performance in Virtual Reality and Real-World Environments

The Effect of Semi-Transparent and Interpenetrable Hands on Object Manipulation in Virtual Reality

Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitation

Contact Info

Product

Resources

About