Interactive Sonification for Data Exploration: How listening modes and display purposes define design guidelines

Abstract

The desire to make data accessible through the sense of listening has led to ongoing research in the fields of sonification and auditory display since the early 1990s. Coming from the disciplines of computer sciences and human computer interface (HCI), the conceptualisation of sonification has been mostly driven by application areas and methods. On the other hand, the sonic arts, which have always participated in the auditory display community, have a genuine focus on sound. Despite these close interdisciplinary relationships between communities of sound practitioners, a rich and sound- or listening-centred concept of sonification is still missing for design guidelines. Complementary to the useful organisation by fields of application, a proper conceptual framework for sound needs to be abstracted from applications and also to some degree from tasks, as both are not directly related to sound. As an initial approach to recasting the thinking about sonification, we propose a conceptualisation of sonifications along two poles in which sound serves either anormativeor adescriptivepurpose. According to these two poles, design guidelines can be developed proper to display purposes and listening modes.

Sonifying data uncertainty with sound dimensions

“…Listening to data has proven useful not only in the arts and music composition, but also in seismology and astrophysics, for example to interpret data streams from the Voyager 2 space probe (The Economist, 2016). While evidence of major scientific discoveries made through sonification remains scant, auditory displays are common in several domains, including environmental monitoring (e.g., Geiger counters), complex system monitoring (e.g., telecom networks and stock market trends), and, perhaps more obviously, in interfaces for vision-impaired users (Grond & Hermann, 2014;Loomis, Golledge, & Klatzky, 1998;Nesbitt & Barrass, 2004).…”

Personalised interactive sonification of musical performance data

“…Based on the responses to what motivated people to change their designs, we see that the purpose of the designs differed across audience members (some were interested in exploring the space of the sounds, while others were interested in specific aesthetic goals). This ties closely with Grond and Hermann's conceptualization for sonifications where the sound can serve different purposes, which motivates different aesthetic design decisions [8].…”

“…In any domain, giving sonification users control over aspects of the sonification can allow users to adapt the sonification for their own needs and interests. Grond and Hermann discuss the importance of knowing the purpose of the sound in a sonification and the listening mode that is required to achieve that purpose [8]. By giving users control of the aesthetic design of a sonification, we take advantage of their individualized listening modes (which may change over time) and their personal goals for what they want to hear.…”

Modes of Listening: Why, How and to What?

“…In some cases, it can be difficult to ascertain where the primary literature ends and the secondary literature begins. This is especially true for sonification, as sonification researchers frequently refer to literature by film scholars such as Michel Chion or composers such as Pierre Schaeffer when publishing in sound studies or musicology journals (Vickers 2012;Grond andHermann 2014). 4.…”

“…Explicit references to taxonomies of listening modes are even more widespread in the scientific community dedicated to sonification research, which aims to systematically explore the use of sound to represent data and convey information. Indeed, the distinction between different modes of listening has been a stable feature of the sonification literature, from some of the founding texts of the community (Gaver 1989;Williams 1994) to much more recent contributions (Vickers 2012;Grond and Hermann 2014). An example is William Gaver's (1989) work on the SonicFinder, an early attempt to use auditory displays as part of a computer interface, which was a forerunner of now-ubiquitous sounds such as those announcing new emails or accompanying the moving of a file into a digital trashcan.…”