Proof of Concept of a Generic Toolkit for Sonification: The Sonification Cell in Ossia Score

Abstract

A popular topic in sonification research is the development of a complete, user-friendly tool for sonification creation. A study of existing attempts at such tools highlights what seems to be the main challenge of this endeavor: exhaustiveness with regards to the great diversity of approaches for auditory display of information. Most tools are designed to allow for selecting among a few typical modalities, but could not really be used to create any kind of sonification. In order to tackle this issue, we proposed a theoretical model of the sonification process, which is intended to take all of its properties as a data observation technique into account. The goal for this model is to be translated into a user interface and programming approach as part of a sonification toolkit. In the present paper, we report our work in creating a proof of concept of such a toolkit using the ossia musical sequencing environment, chosen for its proximity to our objectives in terms of user interaction and library of functionalities. This prototype was tested to recreate two of our previous data sonification works. Most of the specificities of these case studies could be recreated properly, though some of the planned features, notably for grain synthesis, are currently missing from the ossia environment. For our future works, we will consider that the current state of this proof of concept is sufficient to start studying the user experience of sonification designers interacting with the toolkit.

Orchestrating Motion: Real-Time Sonification Tools and Methods to Support Movement Rehabilitation

“…Guidance for novices is correspondingly scattered, and it is hard for new practitioners to learn how to design sonifications; this is especially problematic given how difficult it generally is to correlate design criteria and their actual implementation [186]. These issues can partly be traced to the lack of a common technological framework or tool for data sonification design, a persisting problem within the field [124,177,181].…”

“…Real-time Interactive Data Input and Algorithm Alteration: Many existing tools support real-time ADA / DDA alteration while the dataset is being sonified [23,42,112,136,139,174,177,181,197,248], but several apparently do not [19,35,59,175,244]. Without real-time control, the designer is forced to modify the PMSon in an offline manner (e.g., by modifying a GUI or piece of code and then re-rendering the sound output).…”

“…Existing sonification platforms have either a streamlined GUI, code editor, or some combination thereof. In some tools there is a mapping matrix [23], in most there are multiple tabs with controls to adjust the mappings pertaining to each data variable [136,197,239,244,252], and some even have Pure Datastyle visual connector elements [177,181] or low-level code for maximum flexibility [118,175,187,188,250].…”

“…Only a few tools offer an option to change the shape of the transfer function [12,136,244]. In several recent tools, the transfer function has to be directly programmed in code by the designer [175,181].…”