A methodology for gestural interaction relying on user-defined gestures sets following a one-shot learning approach

How to determine highly effective and intuitive gesture sets for interactive systems tailored to end users’ preferences? A substantial body of knowledge is available on this topic, among which gesture elicitation studies stand out distinctively. In these studies, end users are invited to propose gestures for specific referents, which are the functions to control for an interactive system. The vast majority of gesture elicitation studies conclude with a consensus gesture set identified following a process of consensus or agreement analysis. However, the information about specific gesture sets determined for specific applications is scattered across a wide landscape of disconnected scientific publications, which poses challenges to researchers and practitioners to effectively harness this body of knowledge. To address this challenge, we conducted a systematic literature review and examined a corpus of N = 267 studies encompassing a total of 187,265 gestures elicited from 6,659 participants for 4,106 referents. To understand similarities in users’ gesture preferences within this extensive dataset, we analyzed a sample of 2,304 gestures extracted from the studies identified in our literature review. Our approach consisted of (i) identifying the context of use represented by end users, devices, platforms, and gesture sensing technology, (ii) categorizing the referents, (iii) classifying the gestures elicited for those referents, and (iv) cataloging the gestures based on their representation and implementation modalities. Drawing from the findings of this review, we propose guidelines for conducting future end-user gesture elicitation studies.

show abstract

A grammar for specifying full-body gestures elicited for abstract tasks

Céspedes-Hernández

González-Calleros

Guerrero-García

et al. 2020

IFS

View full text Add to dashboard Cite

A gesture elicitation study consists of a popular method for eliciting a sample of end end users to propose gestures for executing functions in a certain context of use, specified by its users and their functions, the device or the platform used, and the physical environment in which they are working. Gestures proposed in such a study needs to be classified and, perhaps, extended in order to feed a gesture recognizer. To support this process, we conducted a full-body gesture elicitation study for executing functions in a smart home environment by domestic end users in front of a camera. Instead of defining functions opportunistically, we define them based on a taxonomy of abstract tasks. From these elicited gestures, a XML-compliant grammar for specifying resulting gestures is defined, created, and implemented to graphically represent, label, characterize, and formally present such full-body gestures. The formal notation for specifying such gestures is also useful to generate variations of elicited gestures to be applied on-the-fly on gestures in order to allow one-shot learning.

show abstract

A methodology for gestural interaction relying on user-defined gestures sets following a one-shot learning approach

Cited by 3 publications

References 27 publications

Cultural aspects in the user experience design of an ASLR system

Cultural aspects in the user experience design of an ASLR system

Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation Studies

A grammar for specifying full-body gestures elicited for abstract tasks

Contact Info

Product

Resources

About