Designing user-, hand-, and handpart-aware tabletop interactions with the TouchID toolkit

Abstract

Recent work in multi-touch tabletop interaction introduced many novel techniques that let people manipulate digital content through touch. Yet most only detect touch blobs. This ignores richer interactions that would be possible if we could identify (1) which hand, (2) which part of the hand, (3) which side of the hand, and (4) which person is actually touching the surface. Fiduciary-tagged gloves were previously introduced as a simple but reliable technique for providing this information. The problem is that its lowlevel programming model hinders the way developers could rapidly explore new kinds of user-and handpartaware interactions. We contribute the TOUCHID toolkit to solve this problem. It allows rapid prototyping of expressive multi-touch interactions that exploit the aforementioned characteristics of touch input. TOUCHID provides an easy-to-use event-driven API. It also provides higher-level tools that facilitate development: a glove configurator to rapidly associate particular glove parts to handparts; and a posture configurator and gesture configurator for registering new hand postures and gestures for the toolkit to recognize. We illustrate TOUCHID's expressiveness by showing how we developed a suite of techniques (which we consider a secondary contribution) that exploits knowledge of which handpart is touching the surface.

Toward the reduction of incorrect drawn ink retrieval

“…Several researches for the multi-touch interactions [14][15][16] attempt to recognize finger touches. Current capacitive touch devices can correctly receive all touches, but still have a difficulty in classifying which touches are intended and which touches are not.…”

Evaluating heuristics for tabletop user segmentation based on simultaneous interaction

“…However, if they move away, the system forgets about them and they are considered new users when they return. Other works use gloves with coded tags (Marquardt, Kiemer, Ledo, Boring, & Greenberg, 2011), wristbands with LED transmitting an identification code (Meyer & Schmidt, 2010) or rings transmitting infrared light pulses (Roth, Schmidt, & Güldenring, 2010), which are recognized by the vision systems of some tabletops (e.g. Microsoft PixelSense).…”

“…Some drawbacks are identified by their respective authors, such as that the users are tied to a fixed location (Dietz & Leigh, 2001) (Dohse, Dohse, Still, & Parkhurst, 2008), the recognition is not very accurate when several users are working on the same side of the table (Dohse, Dohse, Still, & Parkhurst, 2008), and the number of users is limited (Dietz & Leigh, 2001) (Harrison, Sato, & Poupyrev, 2012) (Marquardt, Kiemer, Ledo, Boring, & Greenberg, 2011) (Meyer & Schmidt, 2010) (Roth, Schmidt, & Güldenring, 2010). Also, the use of wearable hardware (Marquardt, Kiemer, Ledo, Boring, & Greenberg, 2011) (Meyer & Schmidt, 2010) (Roth, Schmidt, & Güldenring, 2010) may result uncomfortable for some people, and it usually requires a previous registration from the users (also observed in (Schmidt, Chong, & Gellersen, 2010)), which is not a very natural way of starting the interactions.…”

A three-step interaction pattern for improving discoverability in finger identification techniques

“…bare hands [4] or fingers with coloured rings [7], wearing gloves with fiducial markers [5], recognizing fingerprints [2], and forearm electromyography [1]. It seems inevitable that one day finger identification will be a standard feature of consumer multi-touch devices.…”

“…Marquardt et al built an exploratory toolkit for tabletops [5] and explored a number of interaction technique ideas to emphasize the toolkit's expressiveness. However, these explorations did not address the problem of discoverability brought by finger identification: discriminating fingers on a touch screen theoretically enables the access to up to 1023 finger combinations, a too large of a set to choose from for one who is not aware of the fingers-to-command mapping.…”