In this paper, we propose an interactive virtual grasping system, enabling physically-realistic interaction between a user and virtual objects, using kinematic hand models whose motions are tracked and controlled in real-time by the user. Since hand tracking devices provide only one-way coupling between the user and kinematic hands resulting in penetrations and friction-less interaction, our method implements the contacts as Coulomb's friction model and includes a dynamic proxy hand in the system which follows the kinematic virtual hand to provide a two-way control loop for the physics interactions. Our system, taking advantage of the graphics and physics engine of Unity3D, presently allows us to interact with 3D objects in the virtual space in real-time with no visible latency. Moreover, to visually assist the user's grasping we render the hand model as transparent and provide visual markers at the contact locations. We demonstrate how the use of these tools helps us to perform real-time grasping tasks in complicated virtual environment.

In this paper, an effective, interactive grasping algorithm is proposed to provide physically realistic interactions in the virtual world. We have provided a solution to lacking dynamic control resulting in penetrations and frictionless contact due to kinematic virtual hand provided by hand-tracking devices, by including a proxy hand in the system. The introduced proxy hand not only provides correct visual feedback but also is used to simulate dynamics between the hand and virtual objects. Our method also provides semiautonomous assistive grasping, coupled with physics-based grasping, which makes the system help the user in achieving a grasp by identifying grasp pose and orienting the object in the hand to be grasped robustly. We have implemented and evaluated our technique with various benchmarks, conducted a user study to compare our method against the state-of-the-art, pinch-based grasping mechanism, and showed that our technique provides physically realistic, stable, and time-efficient real-time interactions.