It’s about Time: Adopting Theoretical Constructs from Visualization for Sonification

Enge, Kajetan; Rind, Alexander; Iber, Michael; Höldrich, Robert; Aigner, Wolfgang

doi:10.1145/3478384.3478415

Cited by 12 publications

(14 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…After all, research in auditory interfaces points to the main diference in perception of visual and sonic information: visual marks are perceived in space and auditory marks in time [12,38]. The latter emphasizes the sequences of sound and speech respectively sound waves over time.…”

Section: Facing Challenges In Interactions With Ipas Through Multimod...mentioning

confidence: 99%

“…In comparison, visual content does not change in perception by looking at it for some time. Of course, the meaning behind what is shown might change, but not the content at present [12]. In general, visualization has the advantage to provide a hierarchy of informational content.…”

Section: Facing Challenges In Interactions With Ipas Through Multimod...mentioning

confidence: 99%

“…Similar to super apps [6], users expected the IPA to orchestrate its main functions and all activated skills. Due to the specifcs of the speech, users perceived the interaction as one auditive stream of information [12] and oftentimes lost their awareness of the skills' scope. Whereas supper apps [6] manage to communicate the state and keep the conversation on track, Alexa Echo Show regularly confused the participants by not operating on consistent commands, for example, "undo" or "skip", as we detailed in Section 4.3.…”

Section: Diferentiation Between Globalmentioning

confidence: 99%

See 2 more Smart Citations

Losing Its Touch: Understanding User Perception of Multimodal Interaction and Smart Assistance

Krauß

Lawo

Stevens

2022

Designing Interactive Systems Conference

View full text Add to dashboard Cite

Intelligent Personal Assistants (IPA) are advertised as reliable companions in the everyday life to simplify household tasks. Due to speech-based usability issues, users struggle to deeply engage with current systems. The capabilities of newer generations of standalone devices are even extended by a display, also to address some weaknesses like memorizing auditive information. So far, it is unclear how the potential of a multimodal experience is realized by designers and appropriated by users. Therefore, we observed 20 participants in a controlled setting, planning a dinner with the help of an audio-visual-based IPA, namely Alexa Echo Show. Our study reveals ambiguous mental models of perceived and experienced device capabilities, leading to confusion. Meanwhile, the additional visual output channel could not counterbalance the weaknesses of voice interaction. Finally, we aim to illustrate users' conceptual understandings of IPAs and provide implications to rethink audiovisual output for voice-frst standalone devices. CCS CONCEPTS• Human-centered computing → Sound-based input / output; Displays and imagers; User studies; Empirical studies in interaction design.

show abstract

Section: Facing Challenges In Interactions With Ipas Through Multimod...mentioning

confidence: 99%

Section: Facing Challenges In Interactions With Ipas Through Multimod...mentioning

confidence: 99%

Section: Diferentiation Between Globalmentioning

confidence: 99%

See 1 more Smart Citation

Losing Its Touch: Understanding User Perception of Multimodal Interaction and Smart Assistance

Krauß

Lawo

Stevens

2022

Designing Interactive Systems Conference

View full text Add to dashboard Cite

show abstract

“…SoniScope es un programa que comenzó su desarrollo recientemente, el trabajo que lo describe data de 2022 (Enge et al, 2022). Esta herramienta produce la sonorización de gráficos de dispersión, la diferencia con otros recursos es la forma en que selecciona que datos sonorizar.…”

Section: Soniscopeunclassified

Evolving from xSonify: a new digital platform for sonorization

et al. 2019

View full text Add to dashboard Cite

In general, Space Sciences data exploration is thought to be suitable for people who do not have Neurological Disorders (ND) that may impair reading (RI), and are not blind or visually impaired (BVI). Data exploration methods used in these fields are generally visual, and do not provide the accessibility to allow full participation of blind and visually impaired individuals in the field. This has been the case even after evidence that sound increases the sensitivity to events in data that would otherwise be undetected. A set of tools that guarantees a thorough data inspection, and assures the access to the same amount and quality of information to all, is useful for everyone. The present project, devoted to design a new user centered software to produce audio-visual outputs from astrophysical data, integrates multidisciplinary and interdisciplinary astronomers, engineers, computer specialists, software designers, educators, disability specialists, bioengineers, neurobiologists, and sociologists, both blind and sighted, and addresses the topics of accessibility to astronomical data, The possibility of exploring the bibliographic databases (ADSABS, among others), creation of a new multimodal access prototype, creation of an accessible, user-centered (from back end to front end), human-computer interface suitable for collection, sonification and analysis of astrophysical data, and the efficiency, utility and effectiveness of the resource in different cultural environments and at different levels of expertise. In this contribution, we present the first results related to the creation and application of the techniques and tools proposed, taking into account that the focus of this research is on the knowledge in the field of physics and astrophysics, but can be scalable to other areas of human activity so that people with other learning styles can have "equal opportunities". Finally, this project is based on the premises of addressing the technical challenges in basic research to achieve the promised benefits of Big Data, which is of world-wide interest.

show abstract

“…Meanwhile, sonification is still an emerging discipline that struggles to define its boundaries, its impacts and more importantly, shared methods, processes, and tools for the mapping of data to sound (Lenzi, 2021 ). The relationship between standardized data visualization strategies such as Bértin's “visual variables” and sonification strategies is a growing Research Topic within the sonification community (Enge et al, 2021 ; Caiola et al, 2022 ).…”

Section: Introductionmentioning

confidence: 99%

Climate data sonification and visualization: An analysis of topics, aesthetics, and characteristics in 32 recent projects

2023

View full text Add to dashboard Cite

IntroductionIt has proven a hard challenge to stimulate climate action with climate data. While scientists communicate through words, numbers, and diagrams, artists use movement, images, and sound. Sonification, the translation of data into sound, and visualization, offer techniques for representing climate data with often innovative and exciting results. The concept of sonification was initially defined in terms of engineering, and while this view remains dominant, researchers increasingly make use of knowledge from electroacoustic music (EAM) to make sonifications more convincing.MethodsThe Aesthetic Perspective Space (APS) is a two-dimensional model that bridges utilitarian-oriented sonification and music. We started with a review of 395 sonification projects, from which a corpus of 32 that target climate change was chosen; a subset of 18 also integrate visualization of the data. To clarify relationships with climate data sources, we determined topics and subtopics in a hierarchical classification. Media duration and lexical diversity in descriptions were determined. We developed a protocol to span the APS dimensions, Intentionality and Indexicality, and evaluated its circumplexity.ResultsWe constructed 25 scales to cover a range of qualitative characteristics applicable to sonification and sonification-visualization projects, and through exploratory factor analysis, identified five essential aspects of the project descriptions, labeled Action, Technical, Context, Perspective, and Visualization. Through linear regression modeling, we investigated the prediction of aesthetic perspective from essential aspects, media duration, and lexical diversity. Significant regressions across the corpus were identified for Perspective (ß = 0.41***) and lexical diversity (ß = −0.23*) on Intentionality, and for Perspective (ß = 0.36***) and Duration (logarithmic; ß = −0.25*) on Indexicality.DiscussionWe discuss how these relationships play out in specific projects, also within the corpus subset that integrated data visualization, as well as broader implications of aesthetics on design techniques for multimodal representations aimed at conveying scientific data. Our approach is informed by the ongoing discussion in sound design and auditory perception research communities on the relationship between sonification and EAM. Through its analysis of topics, qualitative characteristics, and aesthetics across a range of projects, our study contributes to the development of empirically founded design techniques, applicable to climate science communication and other fields.

show abstract

It’s about Time: Adopting Theoretical Constructs from Visualization for Sonification

Cited by 12 publications

References 22 publications

Losing Its Touch: Understanding User Perception of Multimodal Interaction and Smart Assistance

Losing Its Touch: Understanding User Perception of Multimodal Interaction and Smart Assistance

Evolving from xSonify: a new digital platform for sonorization

Climate data sonification and visualization: An analysis of topics, aesthetics, and characteristics in 32 recent projects

Contact Info

Product

Resources

About