Is Sonification Doomed to Fail?

Abstract

Despite persistent research and design efforts over the last twenty years, widespread adoption of sonification to display complex data has largely failed to materialize, and many of the challenges to successful sonification identified in the past persist. Major impediments to the widespread adoption sonification include fundamental perceptual differences between vision and audition, large individual differences in auditory perception, musical biases of sonification researchers, and the interdisciplinary nature of sonification research and design. The historical and often indiscriminate mingling of art and science in sonification design may be a root cause of some of these challenges. Future sonification design efforts that explicitly strive to meet either artistic or scientific goals may lead to greater clarity and success in the field and more widespread adoption of useful sonification techniques.

The sound of migration: exploring data sonification as a means of interpreting multivariate salmon movement datasets

“…In particular, our results indicate that pitch and timbre are the most easily recognized sonic transitions, with volume and panning transitions being recognized slightly less often. This indicates that our naïve participants fall within the expectations of previous research showing that pitch and timbre are effective, and often used, mappings ( Dubus et al., 2013 ; Neuhoff, 2011 ).…”

“…These expressed challenges could be interpreted as the result of an additional data stream which increased processing time, however, the numerous responses indicating that the sound and audio did not match up may indicate another problem. Neuhoff (2011) discusses how visual and audio cues can interact, and that mismatched audio and visual can cause the listener to focus on one stream or the other. The fact that so many respondents felt the visuals did not match the audio indicates some degree of this “ventriloquist effect” in our results that may have increased response time due to increased confusion or switching from audio to visual cues.…”

“…In contrast to visualization, hearing is inherently multidimensional ( Moore, 1995 ) and the human ability to interpret nuanced changes in pattern, and especially timing, in audio signals is striking ( Fitch and Kramer, 1994 ; Kramer et al., 2010 ; Moore, 1995 ; Neuhoff, 2011 ). This is exemplified by the so called, “cocktail party problem,” the observation that human hearing is remarkably capable of disentangling many simultaneous channels of sonic input to focus only on a sound of interest ( McDermott, 2009 ).…”

Tiltification — An Accessible App to Popularize Sonification

“…The idea of a "Killer App" is widely spread within the auditory display community [4,5,6], [7, pp. 105f].…”

“…However, some sonification researchers argue that a killer app is not required for sonification to establish. When enough useful products are brought to the market, sonification will eventually reach the critical mass and become accepted as an alternative or complement of visualization [4,5]. They refer to sonifications that are already established in their fields, like auditory pulse oximetry [8] and the Geiger Counter [9].…”