Gesture elicitation for 3D travel via multi-touch and mid-Air systems for procedurally generated pseudo-universe

Ortega, Francisco R.; Galvan, Alain; Tarre, Katherine; Barreto, Armando; Rishe, Naphtali; Bernal, Jonathan; Balcazar, Ruben; Thomas, Jason-Lee

doi:10.1109/3dui.2017.7893331

Cited by 26 publications

(14 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The gesture proposals were commonly single-handed. This is similar to findings on multi-touch surfaces [28,38,39] and mid-air full-body studies [42]. For manipulations users often interacted based on that actions real-world corollary.…”

Section: Discussionsupporting

confidence: 87%

See 1 more Smart Citation

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Williams

Ortega

2020

Proc. ACM Hum.-Comput. Interact.

Self Cite

View full text Add to dashboard Cite

This research establishes a better understanding of the syntax choices in speech interactions and of how speech, gesture, and multimodal gesture and speech interactions are produced by users in unconstrained object manipulation environments using augmented reality. The work presents a multimodal elicitation study conducted with 24 participants. The canonical referents for translation, rotation, and scale were used along with some abstract referents (create, destroy, and select). In this study time windows for gesture and speech multimodal interactions are developed using the start and stop times of gestures and speech as well as the stoke times for gestures. While gestures commonly precede speech by 81 ms we find that the stroke of the gesture is commonly within 10 ms of the start of speech. Indicating that the information content of a gesture and its co-occurring speech are well aligned to each other. Lastly, the trends across the most common proposals for each modality are examined. Showing that the disagreement between proposals is often caused by a variation of hand posture or syntax. Allowing us to present aliasing recommendations to increase the percentage of users' natural interactions captured by future multimodal interactive systems. CCS Concepts: • Human-centered computing → Human computer interaction (HCI); User studies; Mixed / augmented reality; Interaction techniques; Empirical studies in HCI; User centered design.

show abstract

Section: Discussionsupporting

confidence: 87%

“…Displaying the referent in elicitation studies [42] and reading the referent aloud in gesture and speech elicitation studies [37] both have precedence. These practices can prime the utterances proposed.…”

Section: Speech Proposalsmentioning

confidence: 99%

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Williams

Ortega

2020

Proc. ACM Hum.-Comput. Interact.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Generally, there are many other interesting domains of application for which elicitation studies have been conducted ( Table 3) like mid-air manipulations in Virtual Reality [20,35,48,59], Smart Home environments [7,25,38,46] Mobile Devices [4,43,47,49] for Human-Robot/Drone manipulation [45,57,58], Augmented Reality [33,39], Desktop computer [30,50], In-Vehicle secondary driving task [52,56] Smartwatches [11,32], Gaming [19], CAD [53], Text readers [31], Operating rooms [23] and Digital exhibition [21].…”

Section: Application Domainsmentioning

confidence: 99%

“…A deeper look into the meaning of "appropriateness" in the papers examined ( Table 5), reveals that a considerable number of studies (14 out of 47, 29.8%) investigate a gesture set that is a "better match" or "fit for purpose" for its intended use (without analysing this into a more specific meaning). For example, elicitation studies were conducted to elicit appropriate gesture sets for controlling a media player [54] or for 3D travelling within a pseudo-universe [59], with the aim improve the user interaction and experience in general. Many elicitation studies focus on finding gestures that are easy to perform (12 out of 47, 25.5%).…”

Section: On Gesture "Appropriateness"mentioning

confidence: 99%

Gesture Elicitation Studies for Mid-Air Interaction: A Review

Vogiatzidakis

Koutsabasis

2018

MTI

View full text Add to dashboard Cite

Mid-air interaction involves touchless manipulations of digital content or remote devices, based on sensor tracking of body movements and gestures. There are no established, universal gesture vocabularies for mid-air interactions with digital content or remote devices based on sensor tracking of body movements and gestures. On the contrary, it is widely acknowledged that the identification of appropriate gestures depends on the context of use, thus the identification of mid-air gestures is an important design decision. The method of gesture elicitation is increasingly applied by designers to help them identify appropriate gesture sets for mid-air applications. This paper presents a review of elicitation studies in mid-air interaction based on a selected set of 47 papers published within 2011-2018. It reports on: (1) the application domains of mid-air interactions examined; (2) the level of technological maturity of systems at hand; (3) the gesture elicitation procedure and its variations; (4) the appropriateness criteria for a gesture; (5) participants number and profile; (6) user evaluation methods (of the gesture vocabulary); (7) data analysis and related metrics. This paper confirms that the elicitation method has been applied extensively but with variability and some ambiguity and discusses under-explored research questions and potential improvements of related research.Multimodal Technologies and Interact. 2018, 2, 65 2 of 21 such thing as a universal gesture vocabulary for every application" [15]. Thus, the identification of appropriate gestures for mid-air user interactions in terms of criteria like discoverability, memorability, performance, reliability and comfort, is an important design decision. This must be made at the early stages of system development and it severely affects the development course of every mid-air application project as well as the user experience (UX) of intended users.Emerging from the field of participatory design, gesture elicitation studies have been widely applied to help designers select the most appropriate gesture set for a given application. Although the method does not have a strict procedure, the main approach is to first define what operations (called "referents") have to be executed through gestures, then ask the end users to propose at least one gesture that they find preferable for each referent and finally to extract the gesture vocabulary after analysing the collected data [16]. The goal of a gesture elicitation study is to extract "good" gestures, which according to Morris et al. [17] are "gestures that meet certain design criteria such as discoverability, ease-of-performance, memorability, or reliability."Over the last few years, several elicitation studies have been conducted to identify mid-air gesture vocabularies for many different application domains and contexts, devices, types of digital content and tracking technologies subsumed. A survey of elicitation studies of mid-air interaction is useful to HCI researchers and interaction designers for deepening th...

show abstract

“…Norton et al investigated design strategies for full body gestures in video games by studying human preference when given complete freedom of choosing gestures (Norton et al, 2010). In fact, there is a whole family of user studies called "gesture elicitation" that tries to derive a suitable gesture set from the users themselves (Ortega et al, 2017). Overall, these efforts usually focus on finding the best mapping from a group of gestures to a group of actions required by the task.…”

Section: Gesture-based Interactionmentioning

confidence: 99%

Force Push: Exploring Expressive Gesture-to-Force Mappings for Remote Object Manipulation in Virtual Reality

Bowman

2018

Front. ICT

View full text Add to dashboard Cite

This paper presents Force Push, a novel gesture-based interaction technique for remote object manipulation in virtual reality (VR). Inspired by the design of magic powers in popular culture, Force Push uses intuitive hand gestures to drive physics-based movement of the object. Using a novel algorithm that dynamically maps rich features of hand gestures to the properties of the physics simulation, both coarse-grained ballistic movements and fine-grained refinement movements can be achieved seamlessly and naturally. An initial user study of a limited translation task showed that, although its gesture-to-force mapping is inherently harder to control than traditional position-to-position mappings, Force Push is usable even for extremely difficult tasks. Direct position-to-position control outperformed Force Push when the initial distance between the object and the target was close relative to the required accuracy; however, the gesture-based method began to show promising results when they were far away from each other. As for subjective user experience, Force Push was perceived as more natural and fun to use, even though its controllability and accuracy were thought to be inferior to direct control. This paper expands the design space of object manipulation beyond mimicking reality, and provides hints on using magical gestures and physics-based techniques for higher usability and hedonic qualities in user experience.

show abstract

Gesture elicitation for 3D travel via multi-touch and mid-Air systems for procedurally generated pseudo-universe

Cited by 26 publications

References 37 publications

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Gesture Elicitation Studies for Mid-Air Interaction: A Review

Force Push: Exploring Expressive Gesture-to-Force Mappings for Remote Object Manipulation in Virtual Reality

Contact Info

Product

Resources

About