Towards An In-Vehicle Sonically-Enhanced Gesture Control Interface: A Pilot Study

Abstract

A pilot study was conducted to explore the potential of sonically-enhanced gestures as controls for future in-vehicle information systems (IVIS). Four concept menu systems were developed using a LEAP Motion and Pure Data: (1) 2x2 with auditory feedback, (2) 2x2 without auditory feedback, (3) 4x4 with auditory feedback, and (4) 4x4 without auditory feedback. Seven participants drove in a simulator while completing simple target-acquisition tasks using each of the four prototype systems. Driving performance and eye glance behavior were collected as well as subjective ratings of workload and system preference. Results from driving performance and eye tracking measures strongly indicate that the 2x2 grids yield better driving safety outcomes than 4x4 grids. Subjective ratings show similar patterns for driver workload and preferences. Auditory feedback led to similar improvements in driving performance and eye glance behavior as well as subjective ratings of workload and preference, compared to visual-only.

Novel Multimodal Feedback Techniques for In-Car Mid-Air Gesture Interaction

“…Both studies report different observations regarding eyes-off-the-road time and perceived mental workload. Sterkenburg et al [35] found a significant difference in eyes-off-the-road time and mental demand between the visual and visual-auditory condition, which is contrary to May et al's [25] findings. Thus, research is still required in order to address these questions sufficiently.…”

“…Sterkenburg et al [35] observed significant reductions in eyesoff-the-road time their visual and auditory conditions. They conducted a pilot study on the potential of auditory feedback for in-air gesture control in vehicles.…”

“…Using different sensory channels for mid-air gesture feedback has the potential to enhance in-car interaction, however research has only considered visual and audio-visual feedback [25,35], leaving other modalities unstudied. Thus, this paper presents an investigation into the effects of different types of feedback for in-car mid-air gestures: (1) auditory feedback; (2) peripheral lights feedback; and (3) cutaneous push feedback.…”

May the Force Be with You

“…May et al [32] found that mental demand did not increase nor did driving performance decrease compared to direct touch interaction when auditory and visual feedback were presented bimodally. Sterkenburg et al [43,42] also explored bi-modal audio-visual feedback for gesture control and found that it led to significantly less eyes-off-the-road time and lane deviation than uni-modal visual feedback. However, May et al [32] and Sterkenburg et al [43,48] presented visual feedback in their bi-modal conditions continuing to present information to a channel which is fully occupied by the primary driving task.…”

“…Auditory feedback in driving environments has been shown to reduce looking away time [10] and if presented to mid-air gestures it reduces eyes-off-the-road time without negatively impacting the driving performance [41,43] nor mental demand [41,32]. A multimodal combination of ultrasound and audio feedback can surpass the shortcomings of ultrasound haptics for mid-air feedback [8].…”