Gesture and speech elicitation for 3D CAD modeling in conceptual design

Khan, Sumbul; Tunçer, Bige

doi:10.1016/j.autcon.2019.102847

Cited by 26 publications

(23 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study extends previous works done on multimodal gesture and speech elicitation [27,37]. This extension is seen in the results reported and the methodology used.…”

Section: Gesture and Speech Studiessupporting

confidence: 85%

“…This is different from our choice to examine each input individually. In both studies the referents were shown as animations, however, in this study participants were told that they were interacting with a system whereas Khan et al asked participants to describe the referents to another person via a video chat [27]. The use case of computer-aided design as well as the choice of observing interactions compared to referent descriptions is markedly different, with examples of the referents used there being extrude surface or pan.…”

Section: Gesture and Speech Studiesmentioning

confidence: 98%

“…While a few studies look at gesture and speech inputs have examined mid-air gestures [2,11,20,27,30,37], some only looked at a subset of gesturing such as pointing gestures [7,52], paddling gestures [21], or two dimensional (2D) gestures [35,52]. The work presented here examines any mid-air gesture and / or utterance that a participant feels is appropriate for a given referent.…”

Section: Gesture and Speech Studiesmentioning

confidence: 99%

“…This extension is seen in the results reported and the methodology used. A previous study on interactions for computer-aided design program usage on 2d screens tested both gesture and gesture or speech interactions [27]. In that experiment, gestures were tested independently then gesture with optional speech was tested.…”

Section: Gesture and Speech Studiesmentioning

confidence: 99%

See 3 more Smart Citations

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Williams

Ortega

2020

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

This research establishes a better understanding of the syntax choices in speech interactions and of how speech, gesture, and multimodal gesture and speech interactions are produced by users in unconstrained object manipulation environments using augmented reality. The work presents a multimodal elicitation study conducted with 24 participants. The canonical referents for translation, rotation, and scale were used along with some abstract referents (create, destroy, and select). In this study time windows for gesture and speech multimodal interactions are developed using the start and stop times of gestures and speech as well as the stoke times for gestures. While gestures commonly precede speech by 81 ms we find that the stroke of the gesture is commonly within 10 ms of the start of speech. Indicating that the information content of a gesture and its co-occurring speech are well aligned to each other. Lastly, the trends across the most common proposals for each modality are examined. Showing that the disagreement between proposals is often caused by a variation of hand posture or syntax. Allowing us to present aliasing recommendations to increase the percentage of users' natural interactions captured by future multimodal interactive systems. CCS Concepts: • Human-centered computing → Human computer interaction (HCI); User studies; Mixed / augmented reality; Interaction techniques; Empirical studies in HCI; User centered design.

show abstract

“…This study extends previous works done on multimodal gesture and speech elicitation [27,37]. This extension is seen in the results reported and the methodology used.…”

Section: Gesture and Speech Studiessupporting

confidence: 85%

Section: Gesture and Speech Studiesmentioning

confidence: 98%

Section: Gesture and Speech Studiesmentioning

confidence: 99%

Section: Gesture and Speech Studiesmentioning

confidence: 99%

See 2 more Smart Citations

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Williams

Ortega

2020

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

show abstract

“…The experiment was conducted to elicit speech and gesture preferences for conceptual CAD modeling from the stated professional groups. In related studies, we presented user preferences of gestures (Khan and Tuncer, n.d., 2017; Tunçer and Khan, 2018) and the implementation of a prototype (Khan et al ., 2017).…”

Section: Introductionmentioning

confidence: 99%

Speech analysis for conceptual CAD modeling using multi-modal interfaces: An investigation into Architects’ and Engineers’ speech preferences

Khan

Tunçer

2019

AIEDAM

Self Cite

View full text Add to dashboard Cite

Speech- and gesture-based interfaces for computer-aided design (CAD) modeling must employ vocabulary suitable for target professional groups. We conducted an experiment with 40 participants from architecture and engineering backgrounds to elicit their speech preferences for four CAD manipulation tasks: Scale, Rotate, Copy, and Move. We compiled speech command terms used by participants and analyzed verbalizations based on three analytic themes: the exactness of descriptions, the granularity of descriptions, and the use of CAD legacy terms. We found that participants from both groups used precise and vague expressions in their verbalizations and used a median of three parameters in their verbalizations. Architects used CAD legacy terms more than Engineers in the tasks Scale and Rotate. Based on these findings, we give recommendations for the design of speech- and gesture-based interface for conceptual CAD modeling.

show abstract