A Paradigm Shift in Interactive Computing: Deriving Multimodal Design Principles from Behavioral and Neurological Foundations

Stanney, Kay M.; Samman, Shatha N.; Reeves, Leah; Hale, Kelly S.; Buff, Wendi L.; Bowers, Clint A.; Goldiez, Brian; Nicholson, Denise; Lackey, Stephanie

doi:10.1207/s15327590ijhc1702_7

Cited by 49 publications

(26 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This might explain why participants in this experiment spent more time and (reported) effort on identifying vibration cues, though they performed more accurately. Our result supports the previously proposed guideline, suggesting that if response time is critical, tactile modalities should not be used alone, because then the user will take longer to perceive and attend to information [238].…”

Section: Sound Vs Vibrationsupporting

confidence: 90%

“…Drivers also expected this modality variant to be the most useful in a dynamic real-life driving situation, because they could selectively focus on one modality that was more compatible with driving at a specific moment. Previous work has suggested that multisensory signals are highly able to trigger fast reactions (RT advantage, [238]), and are highly able to capture a person's attention no matter what else he/she happens to be doing at the same time [232]. Therefore, we can conclude that, for presenting time-critical information (e.g.…”

Section: Redundant Combinationmentioning

confidence: 80%

“…• Adding gesture to speech offers speed, high-bandwidth information transfer, flexibility, enhanced error avoidance, and relative ease of use [238].…”

Section: Guidelines On Modality Combinationmentioning

confidence: 99%

“…The message to be presented is highly urgent. (Using multiple modalities redundantly can bring a reaction time advantage, meaning that reaction to multimodal stimuli is faster than to unimodal stimuli [238]. In this specific case, using sound and vibration may allow both a fast response and an accurate priority identification.…”

Section: Sound Vs Vibrationmentioning

confidence: 99%

See 3 more Smart Citations

Multimodal information presentation for high load human computer interaction

Cao¹

View full text Add to dashboard Cite

Section: Sound Vs Vibrationsupporting

confidence: 90%

Section: Redundant Combinationmentioning

confidence: 80%

“…• Adding gesture to speech offers speed, high-bandwidth information transfer, flexibility, enhanced error avoidance, and relative ease of use [238].…”

Section: Guidelines On Modality Combinationmentioning

confidence: 99%

Section: Sound Vs Vibrationmentioning

confidence: 99%

See 2 more Smart Citations

Multimodal information presentation for high load human computer interaction

Cao¹

View full text Add to dashboard Cite

“…The cognitive compatibility issue could be particularly important in these applications, because users are very likely to work under high cognitive load and stress. The need to integrate relevant cognitive knowledge into IMMP has gained awareness in recent years and has been addressed in several articles providing design guidelines [41,45,54].…”

Section: Introductionmentioning

confidence: 99%

Cognitive-Aware Modality Allocation in Intelligent Multimodal Information Presentation

Cao

Theune

Nijholt

2010

Multimedia Interaction and Intelligent User Interfaces

View full text Add to dashboard Cite

Intelligent multimodal presentation (IMMP) systems are able to generate multimodal presentations adaptively, based on the run-time requirements of user-computer interaction. Modality allocation in IMMP system needs to adapt the modality choice to changes in various relevant factors, such as the type of information to be conveyed, the presentation goal, the characteristics of the available modalities, the user profile, the condition of the environment, and the type of user task. In this study, we emphasize that modality allocation in IMMP systems should also take into account the cognitive impacts of modality on human information processing. We first describe several modality-related cognitive and neuropsychological findings. Then a user study is presented to demonstrate the effects of modality on performance, cognitive load and stress, using a high-load and time-critical user task. Finally, we show a possible way to integrate relevant cognitive theories into a computational model that can systematically predict the suitability of a modality choice for a given presentation task.

show abstract