Fig. 1. Snapshots of our online marker-based hand tracking system on sequences with two-handed and hand-object interactions. We demonstrate a novel marker-labeling and tracking system that enables fully-automatic, real-time estimation of hand poses in challenging interaction scenarios with frequent occlusions. Markers labeled as left hand and right hand are rendered as orange and blue spheres respectively, while markers associated with predefined rigid bodies are rendered as green spheres.Optical marker-based motion capture is the dominant way for obtaining high-fidelity human body animation for special effects, movies, and video games. However, motion capture has seen limited application to the human hand due to the difficulty of automatically identifying (or labeling) identical markers on self-similar fingers. We propose a technique that frames the labeling problem as a keypoint regression problem conducive to a solution using convolutional neural networks. We demonstrate robustness of our labeling solution to occlusion, ghost markers, hand shape, and even motions involving two hands or handheld objects. Our technique is equally applicable to sparse or dense marker sets and can run in real-time to support interaction prototyping with high-fidelity hand tracking and hand presence in virtual reality.

We investigate multi-stroke marking menus for multitouch devices and we show that using two hands can improve performance. We present two new two-handed multi-stroke marking menu variants in which users either draw strokes with both hands simultaneously or alternate strokes between hands. In a pair of studies we find that using two hands simultaneously is faster than using a single, dominant-handed marking menu by 10-15%. Alternating strokes between hands doubles the number of accessible menu items for the same number of strokes, and is similar in performance to using a one-handed marking menu. We also examine how stroke direction affects performance. When using thumbs on an iPod Touch, drawing strokes upwards and inwards is faster than other directions. For two-handed simultaneous menus, stroke pairs that are bilaterally symmetric or share the same direction are fastest. We conclude with design guidelines and sample applications to aid multitouch application developers interested in using one-and two-handed marking menus.

No abstract

No abstract

No abstract