Multimodal feedback: an assessment of performance and mental workload

Vitense, Holly S.; Jacko, Julie A.; Emery, Vincent C.

doi:10.1080/00140130303534

Cited by 142 publications

(112 citation statements)

References 17 publications

Supporting

Mentioning

104

Contrasting

Order By: Relevance

“… Furthermore, since the information is delivered sequentially, any interfering effects that could happen in parallel transmission (when two modalities distract from one another) are avoided [87].…”

Section: Advantagesmentioning

confidence: 99%

A Survey of Multimodal Systems and Techniques for Motor Learning

Tadayon

McDaniel²,

Panchanathan

2017

J Inf Process Syst

View full text Add to dashboard Cite

This survey paper explores the application of multimodal feedback in automated systems for motor learning. In this paper, we review the findings shown in recent studies in this field using rehabilitation and various motor training scenarios as context. We discuss popular feedback delivery and sensing mechanisms for motion capture and processing in terms of requirements, benefits, and limitations. The selection of modalities is presented via our having reviewed the best-practice approaches for each modality relative to motor task complexity with example implementations in recent work. We summarize the advantages and disadvantages of several approaches for integrating modalities in terms of fusion and frequency of feedback during motor tasks. Finally, we review the limitations of perceptual bandwidth and provide an evaluation of the information transfer for each modality.

show abstract

Section: Advantagesmentioning

confidence: 99%

A Survey of Multimodal Systems and Techniques for Motor Learning

Tadayon

McDaniel²,

Panchanathan

2017

J Inf Process Syst

View full text Add to dashboard Cite

show abstract

“…On the other hand, our sense of touch is much more limited, with an intensity range of approximately 55dB above the detection threshold. Any vibrations above this threshold feel unpleasant or even painful [150].…”

Section: Intensitymentioning

confidence: 99%

Crossmodal Audio and Tactile Interaction with Mobile Touchscreens

Hoggan

2010

International Journal of Mobile Human Computer Interaction

View full text Add to dashboard Cite

Touchscreen mobile devices often use cut-down versions of desktop user interfaces placing high demands on the visual sense that may prove awkward in mobile settings. The research in this thesis addresses the problems encountered by situationally impaired mobile users by using crossmodal interaction to exploit the abundant similarities between the audio and tactile modalities. By making information available to both senses, users can receive the information in the most suitable way, without having to abandon their primary task to look at the device.This thesis begins with a literature review of related work followed by a definition of crossmodal icons. Two icons may be considered to be crossmodal if and only if they provide a common representation of data, which is accessible interchangeably via different modalities. Two experiments investigated possible parameters for use in crossmodal icons with resultsshowing that rhythm, texture and spatial location are effective.A third experiment focused on learning multi-dimensional crossmodal icons and the extent to which this learning transfers between modalities. The results showed identification rates of 92% for three-dimensional audio crossmodal icons when trained in the tactile equivalents, and identification rates of 89% for tactile crossmodal icons when trained in the audio equivalent.Crossmodal icons were then incorporated into a mobile touchscreen QWERTY keyboard.Experiments showed that keyboards with audio or tactile feedback produce fewer errors and greater speeds of text entry compared to standard touchscreen keyboards. The next study examined how environmental variables affect user performance with the same keyboard. The data showed that each modality performs differently with varying levels of background noise or vibration and the exact levels at which these performance decreases occur were established.The final study involved a longitudinal evaluation of a touchscreen application, CrossTrainer, focusing on longitudinal effects on performance with audio and tactile feedback, the impact of context on performance and personal modality preference. The results show that crossmodal audio and tactile icons are a valid method of presenting information to situationally impaired mobile touchscreen users with recognitions rates of 100% over time.This thesis concludes with a set of guidelines on the design and application of crossmodal audio and tactile feedback to enable application and interface designers to employ such feedback in all systems.

show abstract

“…Thus, a large body of works showed that the audio channel is a very appropriate modality for notification of changes (see [17,24] for examples). Combined with graphics, sound is also supposed to support users in direct manipulation interaction by enhancing the perception of their actions [6,13,20].…”

Section: Enhancing Perceptionmentioning

confidence: 99%

Improving Users’ Comprehension of Changes with Animation and Sound: An Empirical Assessment

Schlienger

Conversy

Chatty

et al. 2007

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Animation or sound is often used in user interfaces as an attempt to improve users' perception and comprehension of evolving situations and support them in decision-making. However, empirical data establishing their real effectiveness on the comprehension of changes are still lacking. We have carried out an experiment using four combinations of visual and auditory feedback in a split attention task. The results not only confirm that such feedback improves the perception of changes, but they also demonstrate that animation and sound used alone or combined bring major improvements on the comprehension of a changing situation. Based on these results, we propose design guidelines about the most efficient combinations to be used in user interfaces.

show abstract

Multimodal feedback: an assessment of performance and mental workload

Cited by 142 publications

References 17 publications

A Survey of Multimodal Systems and Techniques for Motor Learning

A Survey of Multimodal Systems and Techniques for Motor Learning

Crossmodal Audio and Tactile Interaction with Mobile Touchscreens

Improving Users’ Comprehension of Changes with Animation and Sound: An Empirical Assessment

Contact Info

Product

Resources

About