A Study on Gestural Interaction with a 3D Audio Display

Abstract

The study reported here investigates the design and evaluation of a gesturecontrolled, spatially-arranged auditory user interface for a mobile computer. Such an interface may provide a solution to the problem of limited screen space in handheld devices and lead to an effective interface for mobile/eyes-free computing. To better understand how we might design such an interface, our study compared three potential interaction techniques: head nodding, pointing with a finger and pointing on a touch tablet to select an item in exocentric 3D audio space. The effects of sound direction and interaction technique on the browsing and selection process were analyzed. An estimate of the size of the minimum selection area that would allow efficient 3D sound selection is provided for each interaction technique. Browsing using the touch screen was found to be more accurate than the other two techniques, but participants found it significantly harder to use.

A comparison of feedback cues for enhancing pointing efficiency in interaction with spatial audio displays

“…One prominent way of doing this is by providing additional feedback in the form of an external sound source whenever the user is on target. As has been found in our previous work [15,16], this approach is effective and can successfully improve selection speed and accuracy. Such a choice is not unnatural and it is justified by the fact that feedback has to be provided anyway to inform on the current display state, for example to show whether a certain display element is in focus or that it has been selected etc.…”

Sonically-enhanced widgets

“…The position around the head gave the amount downloaded (when the sound reached 12:00 again the download had completed) and rate of movement around the head gave rate of download [Walker and Brewster 2000]. Our work on 3D audio interactions is still continuing, with a focus on the design of basic display elements and interaction techniques [Marentakis and Brewster 2004].…”

Headphones with touch control

“…Metzger et al [2] describe the FreeDigiter, a solution that enables the user to input numbers making finger gestures in front of the earpieces. Other input methods for headphones are described in [3][4] [5]. Also, the use of touch sensing has been previously described in the literature, for example by Hinckley and Sinclair [6] [7], also for detecting contact with the ear [8].…”