Gesture control of sounds in 3D space

Schacher, Jan C.

doi:10.1145/1279740.1279819

Cited by 15 publications

(11 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Therefore, it seems reasonable to regard the means for spatial sound control used in live musical performance from the perspective of design practice of digital musical instruments (DMIs). This potential link has already been roughly explored in previous research (Wanderley and Orio 2002;Marshall et al 2007;Schacher 2007;Perez-Lopez 2015), with a particular focus on the gestural control paradigm. At the core of the DMI metaphor, as introduced by Miranda and Wanderley (2006), stands the decoupling of the physical interface (input or control device) from the sound generating system (contrasting to the integral concept of acoustic musical instruments).…”

Section: Sound Spatialisation Controllers In Context Of Digital Musicmentioning

confidence: 86%

“…There have been a few recent attempts to review the evolution of spatialisation controllers from a historical and musicological perspective (e.g., Brech 2015; Brech and Paland 2015). Some authors have explicitly focused on spatialisation interfaces for real time performances of music (Mooney 2005;Johnson et al , 2014a, others have discussed more recent developments of sound spatialisation systems and spatial rendering frameworks (Marshall et al 2007;Perez-Lopez 2015;Peters 2011;Peters et al 2009;Schacher 2007) as the core component of common software solutions for sound spatialisation. By providing a classification system and a first systematic inventory of spatialisation controllers, our contribution aims at providing deeper insight into design and performance practice of spatial sound controller.…”

Section: A Systematic Inventory Of Spatial Sound Controllers For Realmentioning

confidence: 99%

See 1 more Smart Citation

Instruments for Spatial Sound Control in Real Time Music Performances. A Review

Pysiewicz

Weinzierl

2016

Musical Instruments in the 21st Century

View full text Add to dashboard Cite

The systematic arrangement of sound in space is widely considered as one important compositional design category of Western art music and acoustic media art in the 20th century. A lot of attention has been paid to the artistic concepts of sound in space and its reproduction through loudspeaker systems. Much less attention has been attracted by live-interactive practices and tools for spatialisation as performance practice. As a contribution to this topic, the current study has conducted an inventory of controllers for the real time spatialisation of sound as part of musical performances, and classified them both along different interface paradigms and according to their scope of spatial control. By means of a literature study, we were able to identify 31 different spatialisation interfaces presented to the public in context of artistic performances or at relevant conferences on the subject. Considering that only a small proportion of these interfaces combines spatialisation and sound production, it seems that in most cases the projection of sound in space is not delegated to a musical performer but regarded as a compositional problem or as a separate performative dimension. With the exception of the mixing desk and its fader board paradigm as used for the performance of acousmatic music with loudspeaker orchestras, all devices are individual design solutions developed for a specific artistic context. We conclude that, if controllers for sound spatialisation were supposed to be perceived as musical instruments in a narrow sense, meeting certain aspects of instrumentality, immediacy, liveness, and learnability, new design strategies would be required.

show abstract

Section: Sound Spatialisation Controllers In Context Of Digital Musicmentioning

confidence: 86%

Section: A Systematic Inventory Of Spatial Sound Controllers For Realmentioning

confidence: 99%

Instruments for Spatial Sound Control in Real Time Music Performances. A Review

Pysiewicz

Weinzierl

2016

Musical Instruments in the 21st Century

View full text Add to dashboard Cite

show abstract

“…In the second, performance movements are used to trigger spatialization as in the case of indirect control of spatialization through dancers' movements [Wijnans 2010]. Schacher [2007] distinguishes the latter two approaches as top-down and bottom-up control of spatialization.…”

Section: Interaction Design Aspectsmentioning

confidence: 99%

Perceptual impact of gesture control of spatialization

Marentakis

McAdams

2013

ACM Trans. Appl. Percept.

View full text Add to dashboard Cite

In two experiments, visual cues from gesture control of spatialization were found to affect auditory movement perception depending on the identifiability of auditory motion trajectories, the congruency of audiovisual stimulation, the sensory focus of attention, and the attentional process involved. Visibility of the performer's gestures improved spatial audio trajectory identification, but it shifted the listeners' attention to vision, impairing auditory motion encoding in the case of incongruent stimulation. On the other hand, selectively directing attention to audition resulted in interference from the visual cues for acoustically ambiguous trajectories. Auditory motion information was poorly preserved when dividing attention between auditory and visual movement feedback from performance gestures. An auditory focus of attention is a listener strategy that maximizes performance, due to the improvement caused by congruent visual stimulation and its robustness to interference from incongruent stimulation for acoustically unambiguous trajectories. Attentional strategy and auditory motion calibration are two aspects that need to be considered when employing gesture control of spatialization.

show abstract

“…As in our previous works (Kontogeorgakopoulos, Kotsifa and Erichsen 2011), key motion features such us velocity, presence, position, orientation and acceleration were detected and tracked; in this work, we limited the use of sensors to a single camera-based motion-tracking system. Camera-based motion tracking is not uncommon in interactive music, art and design (Levin 2006; Schacher 2010; Wechsler, Weiss and Dowling 2004; Winkler 1997). Among several existing technological solutions (MAX MSP Jitter with cv.jit library, the Eyesweb platform, the Processing programming language with the openCV library and the openFrameworks framework with the ofxOpenCV), the Eyecon system was utilised.…”

Section: Technologymentioning

confidence: 99%