Investigating Perceptual Congruence between Data and Display Dimensions in Sonification

Abstract

The relationships between sounds and their perceived meaning and connotations are complex, making auditory perception an important factor to consider when designing sonification systems. Listeners often have a mental model of how a data variable should sound during sonification and this model is not considered in most data:sound mappings. This can lead to mappings that are difficult to use and can cause confusion. To investigate this issue, we conducted a magnitude estimation experiment to map how roughness, noise and pitch relate to the perceived magnitude of stress, error and danger. These parameters were chosen due to previous findings which suggest perceptual congruency between these auditory sensations and conceptual variables. Results from this experiment show that polarity and scaling preference are dependent on the data:sound mapping. This work provides polarity and scaling values that may be directly utilised by sonification designers to improve auditory displays in areas such as accessible and mobile computing, process-monitoring and biofeedback.

EEG-based Evaluation of Cognitive Workload Induced by Acoustic Parameters for Data Sonification

“…On the whole, our findings mirror those in [9] and [10]. The best performing acoustic parameter in terms of data:sound mapping accuracies is the combined rendering of roughness and noise (Section 4.1), which owing to its negative attributes best conveys image blur (inverse of focus level).…”

“…An extension of the above study is presented in [10]. Here, the ability of parameters such as roughness, noise and pitch for conveying negative data attributes such as error, danger and stress is explored.…”

“…In [9], a study evaluating auditory parameters (such as pitch, roughness, noise and sharpness) that best convey the focus level of an astronomical image is presented. This study is further extended in [10], where the perceptual congruence between three data attributes, namely, stress, error and danger and the aforementioned acoustic parameters is evaluated. Both evaluations are based on explicit user assessments acquired via the mouse and keyboard.…”

“…Such studies have however not been performed, to our knowledge, for auditory perception tasks. This paper builds on prior works [9,10], and seeks to estimate the cognitive load of users mapping acoustic parameters to data attributes via implicit means. Cognitive load induced by different auditory parameters is examined via explicit as well as implicit user responses-participants completed (a subset of questions from) the NASA-TLX questionnaire [11] to convey their impressions regarding how easy (low cognitive load condition) or difficult (high cognitive load condition) it is to map the focus level of an astronomical image to an acoustic parameter; their EEG responses while performing the mapping task were also recorded via the commercial Emotiv device.…”

“…However, while evaluations of visual tools for their interpretability and intuitiveness have been commonplace, not much research has been devoted to other forms of data rendering, e.g., sonification. This work is the first to automatically estimate the cognitive load induced by different acoustic parameters considered for sonification in prior studies [9,10]. We examine cognitive load via (a) perceptual data-sound mapping accuracies of users for the different acoustic parameters, (b) cognitive workload impressions explicitly reported by users, and (c) their implicit EEG responses compiled during the mapping task.…”