Physics-based Interactive Virtual Grasping

Abstract

In this paper, we propose an interactive virtual grasping system, enabling physically-realistic interaction between a user and virtual objects, using kinematic hand models whose motions are tracked and controlled in real-time by the user. Since hand tracking devices provide only one-way coupling between the user and kinematic hands resulting in penetrations and friction-less interaction, our method implements the contacts as Coulomb's friction model and includes a dynamic proxy hand in the system which follows the kinematic virtual hand to provide a two-way control loop for the physics interactions. Our system, taking advantage of the graphics and physics engine of Unity3D, presently allows us to interact with 3D objects in the virtual space in real-time with no visible latency. Moreover, to visually assist the user's grasping we render the hand model as transparent and provide visual markers at the contact locations. We demonstrate how the use of these tools helps us to perform real-time grasping tasks in complicated virtual environment.

Automatic Detection of Myocontrol Failures Based upon Situational Context Information

“…3) Object Interaction: Physics engines are unstable when multiple contacts act on an object, such as in grasping (e.g. [11]). We thus rely on the physics engine only for collision detection, but allow penetration and specify custom rules for grasp stability.…”

Physics‐based assistive grasping for robust object manipulation in virtual reality

“…The method involves semiautonomous grasp planning as the system detects the first step of planning, that is, establish a valid grasp contact and then allow the user to close the fingers around the object to grab it. We utilize proxy hand and physics‐based contacts to force fingers not to pass through the object and avoid slipping.…”

“…Achieving stable and physically realistic interactions in the VR is a challenging problem. First of all, virtual environment cannot physically constrain real hand motion; the user can not actually feel the object, that is, one‐way coupling between the user and the kinematic hand, resulting in hand–object interpenetrations and frictionless contact. Also, the human hand's high degree of freedom and anatomical structure coupled with the space of grasping parameters and the geometry of the objects create a high‐dimensional problem in configuration space.…”

“…This research is an extension of a preliminary version involving dynamic proxy hand to address hand–object interpenetrations. The same proxy hand algorithm is used here, but the grasp algorithm is extended for the ease of the user by detecting grasp pose and manipulating objects accordingly; also, the system is evaluated by conducting a user study.…”

Efficient Physics-Based Implementation for Realistic Hand-Object Interaction in Virtual Reality

“…Other methods require some prior training phase [34,37]. Physics-based methods, like ours, provide more unconstrained interaction possibilities [4,16,17,27]. However, currently, these methods are mostly limited to simple grasping interactions.…”