Evaluating music performance and context-sensitivity with Immersive Virtual Environments

Glowinski, Donald; Baron, Naëm; Shirole, Kanika; Coll, Sélim Yahia; Chaabi, Lina; Ott, Tamara; Rappaz, Marc-André; Grandjean, Didier

doi:10.4108/ct.2.2.e3

Cited by 6 publications

(7 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Drawing upon the recent studies [70,71], we considered two types of expressive body features: the kinetic energy and the Body Twist Index (BTI). The former helps understand broad, unrefined body reactions and gives a glimpse of the behavioral components of expressive gestures.…”

Section: Multi-modal Expressive Behavior Analysismentioning

confidence: 99%

Playing for a Virtual Audience: The Impact of a Social Factor on Gestures, Sounds and Expressive Intents

2017

Self Cite

View full text Add to dashboard Cite

Can we measure the impact of the presence of an audience on musicians' performances? By exploring both acoustic and motion features for performances in Immersive Virtual Environments (IVEs), this study highlights the impact of the presence of a virtual audience on both the performance and the perception of authenticity and emotional intensity by listeners. Gestures and sounds produced were impacted differently when musicians performed at different expressive intents. The social factor made features converge towards values related to a habitual way of playing regardless of the expressive intent. This could be due to musicians' habits to perform in a certain way in front of a crowd. On the listeners' side, when comparing different expressive conditions, only one congruent condition (projected expressive intent in front of an audience) boosted the participants' ratings for both authenticity and emotional intensity. At different values for kinetic energy and metrical centroid, stimuli recorded with an audience showed a different distribution of ratings, challenging the ecological validity of artificially created expressive intents. Finally, this study highlights the use of IVEs as a research tool and a training assistant for musicians who are eager to learn how to cope with their anxiety in front of an audience.

show abstract

Section: Multi-modal Expressive Behavior Analysismentioning

confidence: 99%

Playing for a Virtual Audience: The Impact of a Social Factor on Gestures, Sounds and Expressive Intents

2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…The second concert focused on the concept of presence by dividing the audience into three groups, namely, a group watching a normal livestream, another watching a livestream with Youtube’s 360° option, and a group watching with a virtual reality (VR) headset. The ability to feel present inside a digital world (e.g., as if the computer interface disappears) has received increasing interest from researchers ( Västfjäll, 2003 ; Sanchez-Vives and Slater, 2005 ; Riva et al, 2007 ; Takatalo et al, 2008 ; Slater et al, 2009 ; Glowinski et al, 2015 ; Çamcı and Hamilton, 2020 ). While a connection between presence and togetherness has been theorized before ( Durlach and Slater, 2000 ), the aim of the current study was to investigate how feelings of presence contribute to feeling socially connected during a livestreamed concert.…”

Section: Introductionmentioning

confidence: 99%

Livestream Experiments: The Role of COVID-19, Agency, Presence, and Social Context in Facilitating Social Connectedness

et al. 2021

View full text Add to dashboard Cite

Musical life became disrupted in 2020 due to the COVID-19 pandemic. Many musicians and venues turned to online alternatives, such as livestreaming. In this study, three livestreamed concerts were organized to examine separate, yet interconnected concepts—agency, presence, and social context—to ascertain which components of livestreamed concerts facilitate social connectedness. Hierarchical Bayesian modeling was conducted on 83 complete responses to examine the effects of the manipulations on feelings of social connectedness with the artist and the audience. Results showed that in concert 1, where half of the participants were allowed to vote for the final song to be played, this option did not result in the experience of more agency. Instead, if their preferred song was played (regardless of voting ability) participants experienced greater connectedness to the artist. In concert 2, participants who attended the concert with virtual reality headsets experienced greater feelings of physical presence, as well as greater feelings of connectedness with the artist, than those that viewed a normal YouTube livestream. In concert 3, attendance through Zoom led to greater experience of social presence, but predicted less connectedness with the artist, compared to a normal YouTube livestream. Crucially, a greater negative impact of COVID-19 (e.g., loneliness) predicted feelings of connectedness with the artist, possibly because participants fulfilled their social needs with this parasocial interaction. Examining data from all concerts suggested that physical presence was a predictor of connectedness with both the artist and the audience, while social presence only predicted connectedness with the audience. Correlational analyses revealed that reductions in loneliness and isolation were associated with feelings of shared agency, physical and social presence, and connectedness to the audience. Overall, the findings suggest that in order to reduce feelings of loneliness and increase connectedness, concert organizers and musicians could tune elements of their livestreams to facilitate feelings of physical and social presence.

show abstract

“…They created real-life virtual settings in which to investigate various topics, such as music therapy (Orman, 2004;Bissonnette et al, 2016), music education (Orman et al, 2017;Serafin et al, 2017), music performance (Williamon et al, 2014;Glowinski et al, 2015), and the relation between sound and presence (Västfjäll, 2003;Kern and Ellermeier, 2020;Kobayashi et al, 2020). A good example of simulating a real-life setting is given by Glowinski et al (2015), who investigated the influence of social context on performance. Specifically, Glowinski and colleagues asked participants to perform a musical task in a virtual concert hall while controlling for audience gaze.…”

Section: Vr: a Research Tool For Studying Social Interactionsmentioning

confidence: 99%

A Methodological Framework for Assessing Social Presence in Music Interactions in Virtual Reality

2021

View full text Add to dashboard Cite

Virtual reality (VR) brings radical new possibilities to the empirical study of social music cognition and interaction. In the present article, we consider the role of VR as a research tool, based on its potential to create a sense of “social presence”: the illusory feeling of being, and socially interacting, inside a virtual environment. This makes VR promising for bridging ecological validity (“research in the wild”) and experimental control (“research in the lab”) in empirical music research. A critical assumption however is the actual ability of VR to simulate real-life social interactions, either via human-embodied avatars or computer-controlled agents. The mediation of social musical interactions via VR is particularly challenging due to their embodied, complex, and emotionally delicate nature. In this article, we introduce a methodological framework to operationalize social presence by a combination of factors across interrelated layers, relating to the performance output, embodied co-regulation, and subjective experiences. This framework provides the basis for the proposal of a pragmatic approach to determine the level of social presence in virtual musical interactions, by comparing the outcomes across the multiple layers with the outcomes of corresponding real-life musical interactions. We applied and tested this pragmatic approach via a case-study of piano duet performances of the piece Piano Phase composed by Steve Reich. This case-study indicated that a piano duet performed in VR, in which the real-time interaction between pianists is mediated by embodied avatars, might lead to a strong feeling of social presence, as reflected in the measures of performance output, embodied co-regulation, and subjective experience. In contrast, although a piano duet in VR between an actual pianist and a computer-controlled agent led to a relatively successful performance output, it was inadequate in terms of both embodied co-regulation and subjective experience.

show abstract

Evaluating music performance and context-sensitivity with Immersive Virtual Environments

Cited by 6 publications

References 44 publications

Playing for a Virtual Audience: The Impact of a Social Factor on Gestures, Sounds and Expressive Intents

Playing for a Virtual Audience: The Impact of a Social Factor on Gestures, Sounds and Expressive Intents

Livestream Experiments: The Role of COVID-19, Agency, Presence, and Social Context in Facilitating Social Connectedness

A Methodological Framework for Assessing Social Presence in Music Interactions in Virtual Reality

Contact Info

Product

Resources

About