Performance effects of multi-sensory displays in virtual teleoperation environments

Abstract

Multi-sensory displays provide information to users through multiple senses, not only through visuals. They can be designed for the purpose of creating a more-natural interface for users or reducing the cognitive load of a visual-only display. However, because multi-sensory displays are often application-specific, the general advantages of multi-sensory displays over visual-only displays are not yet well understood. Moreover, the optimal amount of information that can be perceived through multisensory displays without making them more cognitively demanding than visual-only displays is also not yet clear. Last, the effects of using redundant feedback across senses on multisensory displays have not been fully explored. To shed some light on these issues, this study evaluates the effects of increasing the amount of multi-sensory feedback on an interface, specifically in a virtual teleoperation context. While objective data showed that increasing the number of senses in the interface from two to three led to an improvement in performance, subjective feedback indicated that multi-sensory interfaces with redundant feedback may impose an extra cognitive burden on users.

Multi-Sensory Urban Search-and-Rescue Robotics: Improving the Operator’s Omni-Directional Perception

“…In the analysis of the sketchmap quality results (Figure 10A), trends indicated improvements caused by Interface 2 (W = 11.0, Z = −1.616, p = 0.111, R = −0.165; η 2 = 0.169), and Interface 4 (W = 13.0, Z = −1.633, p = 0.109, R = −0.167; η 2 = 0.057), which partially confirms H3.3. The results go in hand with results obtained previously (Moss et al, 2003;de Barros and Lindeman, 2013), and Wickens' Multiple Resource Theory (Wickens, 2002).…”

“…Results from previous studies suggest that vibro-tactile feedback leads to positive results when used either as a redundant (study #1) or complementary (study #2) type of feedback. This third study builds upon the previous study results, and evaluates the effect of adding extra audio feedback to the previous bi-sensory interface (vision and touch), and the effect of redundant data presentation in multi-sensory displays (de Barros and Lindeman, 2013).…”

“…The second study (de Barros and Lindeman, 2012) explored the pros and cons of two vibro-tactile data display representations. The third study (de Barros and Lindeman, 2013) further enhanced the visual-tactile interface from study #2 with audio and redundant visual feedback, and measured the effects of such enhancements to the interface. The fourth and last study (de Barros, 2014) attempted not only to validate previous results obtained via simulation with a real robot but also evaluated the addition of smell feedback on top of the other three-previously evaluated types of sensory feedback.…”

Feeling Uncertain—Effects of a Vibrotactile Belt that Communicates Vehicle Sensor Uncertainty

“…Apart from its potential for reliability communication, vibrotactile interfaces have been identified as promising elements of user interfaces [24] and particularly applicable in the context of driver assistance [25] such as for driving- [26][27][28][29][30] or navigation support [31][32][33][34][35][36][37][38][39]. In addition, also advanced tactile encodings of relevant information such as spatial distances [40][41][42][43][44][45][46], directions [32,[47][48][49][50][51][52] and spatio-temporal measures [53,54] have been investigated.…”

A New UGV Teleoperation Interface for Improved Awareness of Network Connectivity and Physical Surroundings

“…Perhaps, a better measure is the number of collisions per path length as used in [43]. However, since there were many participants that had no collisions in both groups, it would not be possible to have a fair comparison with the collisions/meter metric.…”