Towards Many Gestures to One Command: A User Study for Tabletops

Abstract

Abstract. Multi-touch gestures are often thought by application designers for a one-to-one mapping between gestures and commands, which does not take into account the high variability of user gestures for actions in the physical world; it can also be a limitation that leads to very simplistic interaction choices. Our motivation is to make a step toward many-to-one mappings between user gestures and commands, by understanding user gestures variability for multi-touch systems; for doing so, we set up a user study in which we target symbolic gestures on tabletops. From a first phase study we provide qualitative analysis of user gesture variability; we derive this analysis into a taxonomy of user gestures, that is discussed and compared to other existing taxonomies. We introduce the notion of atomic movement; such elementary atomic movements may be combined throughout time (either sequentially or in parallel), to structure user gesture. A second phase study is then performed with specific class of gesture-drawn symbols; from this phase, and according to the provided taxonomy, we evaluate user gesture variability with a fine grain quantitative analysis. Our findings indicate that users equally use one or two hands, also that more than half of gestures are achieved using parallel or sequential combination of atomic movements. We also show how user gestures distribute over different movement categories, and correlate to the number of fingers and hands engaged in interaction. Finally, we discuss implications of this work to interaction design, practical consequences on gesture recognition, and potential applications.

Touch interaction for children aged 3 to 6 years: Experimental findings and relationship to motor skills

“…Recognition of touch gestures has been done with algorithms specifically designed to discriminate between single-and multi-stroke gestures (Anthony and Wobbrock, 2010;Rekik et al, 2014;Vatavu, 2012;Vatavu et al, 2012;Wobbrock et al, 2007), and by employing multi-touch gesture toolkits Li, 2012, 2013). Researchers have also looked into users' gesture articulation patterns to understand more about their users and, consequently, improve the accuracy of existing recognition techniques (Anthony et al, 2013b;Rekik et al, 2013;Vatavu et al, 2013;Tu et al, 2012). For instance, Tu et al (2012) reported articulation differences between pen and finger stroke gestures.…”

“…Also, touch input allows a multitude of possible outcomes for finger actions, ranging from simple taps, pinches, and swipes up to complex gestures with meaningful mappings to application functions (Hinrichs and Carpendale, 2011;Rekik et al, 2013;Wobbrock et al, 2009). Extra input modalities were also employed to accompany touch input and increase its expressibility, such as finger pressure (Heo and Lee, 2012;Hennecke et al, 2011) and sensing different parts of the finger anatomy (Harrison et al, 2011b).…”

“…Hinrichs and Carpendale (2011) showed that users' gesture preferences are influenced by the interaction and social context in which they occur, and that gestures are linked to previous articulations and form consistent patterns. Rekik et al (2013) reported the many ways in which users vary their multi-touch articulations when not constrained by the device or the recognizer's sensing capabilities, and proposed a taxonomy to characterize such variations based on the concept of atomic movements. Anthony et al (2013b) analyzed user consistency in gesture production and reported high degree of agreement within-users, lower agreement between-users, and higher consistency for some gesture types.…”

Understanding Users' Perceived Difficulty of Multi-Touch Gesture Articulation

“…Although the notion of gesture articulation difficulty has been mentioned as design criteria by many works [11,13,22], it has only been examined thoroughly for unistrokes [20]. However, opposed to unistrokes, multitouch gestures present considerably more degrees of freedom afforded during articulation, such as employing various number of fingers, different stroke count and ordering, and single-handed and bimanual input [15]. In this work, we examine users' perceived difficulty of multi-touch gesture articulation for symbolic gesture types [3,15,[19][20][21], such as those from Figure 1, and we provide the community the first understanding of this phenomenon as well as its potential implications in terms of gesture set design, multitouch recognizer development, and gesture-to-function mappings.…”

“…Designing gesture sets can prove a challenging task because of users' different preferences in terms of articulation patterns [2,15,16] and gesture-to-function mappings [22]. The current practice of gesture interface design has outlined several guidelines to assist Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”