Toward Augmented Familiarity of the Audience with Digital Musical Instruments

Capra, Olivier; Berthaut, Florent; Grisoni, Laurent

doi:10.1007/978-3-030-01692-0_37

Cited by 3 publications

(4 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Visual feedback can be projected behind the performers [47], overlapped with the instrument using an AR display [8], or provided both on stage and on mobile devices [6]. In a precedent work, we proposed a pipeline based on the integration of physiological data to select relevant visual augmentation [15]. Haptic feedback can also be used to amplify musicians' gestures [1,58].…”

Section: Related Work Spectator Experience Augmentation Techniques (Smentioning

confidence: 99%

“…Beyond such subjective reports, physiological measurement can provide objective insights like the level of engagement of spectators watching dance performances [41]. The integration of physiological sensors is a promising way to gather objective data about the audience experience [60,15,55]. However, the time resolution, the many sources of interference and the variety of interpretation of physiological data are still obstacles to robust applications.…”

Section: Measuring the Audience Experiencementioning

confidence: 99%

See 1 more Smart Citation

Have a SEAT on Stage

Capra

Berthaut

Grisoni

2020

Proceedings of the 2020 ACM Designing Interactive Systems Conference

Self Cite

View full text Add to dashboard Cite

When the collaboration between humans and machines happens in public, the audience can face difficulties in distinguishing the actual human contribution from the contribution of autonomous processes. In music concerts involving digital interfaces doubts about the performer's contribution can drastically hinder the audience interest. The disappearing of the direct physical link between actions and effects is one of the reasons of this confusion. Consequently both artists and researchers have explored techniques to augment the experience of spectators. However their respective impact on the multiple aspects of audience experience has not yet been formally compared. In this controlled study, we compare two techniques : pre-performance explanations and visual augmentations. Despite contradictory results on comprehension tasks, we show that contrary to pre-performance explanations, visual augmentations improve the audience experience, increase their subjective comprehension and restore the trust in performers by reversing the doubt in their favour.

show abstract

Section: Related Work Spectator Experience Augmentation Techniques (Smentioning

confidence: 99%

Section: Measuring the Audience Experiencementioning

confidence: 99%

Have a SEAT on Stage

Capra

Berthaut

Grisoni

2020

Proceedings of the 2020 ACM Designing Interactive Systems Conference

Self Cite

View full text Add to dashboard Cite

show abstract

“…For DMIs in general, the causal link between the performer's gestures and the sound generation may not always be clearly perceivable by the audience, and may impact negatively on the relationship between performers and audience. Augmenting the audience experience through additional sensory channels (e.g., visuals [33] or haptics [34]) can help reestablish such a link by increasing the transparency of the interaction. This issue is even more challenging in the case of HeaDMIs, where mappings have necessarily a limited ecological validity, and transparency from the audience perspective is reduced by the lack of apparent exchange of energy between the performer and the instrument.…”

Section: E Evaluationmentioning

confidence: 99%

Hands-Free Accessible Digital Musical Instruments: Conceptual Framework, Challenges, and Perspectives

Davanzo

Avanzini

2020

IEEE Access

View full text Add to dashboard Cite

Exponential increases of available computational resources, miniaturization, and sensors, are enabling the development of digital musical instruments that use non-conventional interaction paradigms and interfaces. This scenario opens up new opportunities and challenges in the creation of accessible instruments to include persons with disabilities into music practice. This work focuses in particular on instruments dedicated to people who can not use limbs, for whom the only means for musical expression are the voice and a small number of traditional instruments. First, a modular and adaptable conceptual framework is discussed for the design of accessible digital musical instruments targeted at performers with motor impairments. Physical interaction channels available from the neck upwards (head, mouth, eyes, brain) are analyzed in terms of potential and limitations for musical interaction. Second, a systematic survey of previously developed instruments is presented: each is analyzed in terms of design choices, physical interaction channels and related sensors, mapping strategies, performer interface and feedback. As a result of this survey, several open research directions are discussed, including the use of unconventional interaction channels, musical control mappings, multisensory feedback, design, evaluation, and adaptation. INDEX TERMS Accessibile interfaces, Music technology, Digital musical instruments, Inclusive music practice VOLUME ...

show abstract

“…Benford et al (2018) go beyond fixed information by combining projected visual augmentation during the performance, as well as visual and textual augmentation on a mobile app after the performance, thus allowing spectators to access two different levels of representation. Finally, Capra, Berthaut, and Grisoni (2018) propose adaptive augmentation as part of a pipeline for augmented familiarity, but they do not provide an implementation or evaluate the impact of the described levels. In contrast, we work with adapting the amount and the type of information provided by visual augmentation using an LOD approach.…”

Section: Augmenting the Audience Experiencementioning

confidence: 99%

Levels of Detail in Visual Augmentation for Novice and Expert Audiences

Capra

Berthaut

Grisoni

2020

Computer Music Journal

Self Cite

View full text Add to dashboard Cite

Digital musical instruments offer countless opportunities for musical expression by allowing artists to produce sound without the physical constraints of analog instruments. By breaking the intuitive link between gestures and sound they may hinder the audience experience, however, making the musician's contribution and expressiveness difficult to discern. To cope with this issue without altering the instruments, researchers and artists have designed techniques to augment their performances with additional information, through audio, haptic, or visual modalities. These techniques have, however, only been designed to offer a fixed level of information, without taking into account the variety of spectators' expertise and preferences. In this article, we introduce the concept of controllable levels of details (LODs) for visual augmentation. We investigate their design, implementation, and effect on objective and subjective aspects of audience experience. We conduct a controlled experiment with 18 participants, including novices and experts in electronic music. Our results expose the subjective nature of expertise and its biases. We analyze quantitative and qualitative data to reveal contrasts in the impact of LODs on experience and comprehension for experts and novices. Finally, we highlight the diversity of usage of LODs in visual augmentation by spectators and propose a new role on stage, the augmenter.

show abstract

Toward Augmented Familiarity of the Audience with Digital Musical Instruments

Cited by 3 publications

References 23 publications

Have a SEAT on Stage

Have a SEAT on Stage

Hands-Free Accessible Digital Musical Instruments: Conceptual Framework, Challenges, and Perspectives

Levels of Detail in Visual Augmentation for Novice and Expert Audiences

Contact Info

Product

Resources

About