AI-terity 2.0: An Autonomous NIME Featuring GANSpaceSynth Deep Learning Model

Tahiroğlu, Koray; Kastemaa, Miranda; Koli, Oskar

doi:10.21428/92fbeb44.3d0e9e12

Cited by 4 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Over the past decades, the concomitant advances in wearable sensing technologies, such as Inertial Measurement Unit (IMU) sensors, and sound synthesis techniques have paved the way to the development of motion-sound interactive systems. Designing the mapping between movement and sound is essential for interactive audio applications and extensively studied in the New Interfaces for Musical Expression (NIME) community, involving digital musical instrument design [25,35,49] possibly using machine learning [21,30,45].…”

Section: Motion-sound Interactive Systemsmentioning

confidence: 99%

Embodied exploration of deep latent spaces in interactive dance-music performance

Nabi,

Esling,

Peeters

et al. 2024

Proceedings of the 9th International Conference on Movement and Computing

View full text Add to dashboard Cite

Section: Motion-sound Interactive Systemsmentioning

confidence: 99%

Embodied exploration of deep latent spaces in interactive dance-music performance

Nabi,

Esling,

Peeters

et al. 2024

Proceedings of the 9th International Conference on Movement and Computing

View full text Add to dashboard Cite

“…Synthesizers can be very expressive instruments, whether controlled by the ubiquitous keyboard [28], by augmented instruments, or instrument-like interfaces [1,25,26], or by whole new sets of gestures enabled by novel controllers [8,35].…”

Section: Introductionmentioning

confidence: 99%

FM Tone Transfer with Envelope Learning

Caspe,

McPherson,

Sandler

2023

Proceedings of the 18th International Audio Mostly Conference

View full text Add to dashboard Cite

Tone Transfer is a novel deep-learning technique for interfacing a sound source with a synthesizer, transforming the timbre of audio excerpts while keeping their musical form content. Due to its good audio quality results and continuous controllability, it has been recently applied in several audio processing tools. Nevertheless, it still presents several shortcomings related to poor sound diversity, and limited transient and dynamic rendering, which we believe hinder its possibilities of articulation and phrasing in a real-time performance context.In this work, we present a discussion on current Tone Transfer architectures for the task of controlling synthetic audio with musical instruments and discuss their challenges in allowing expressive performances. Next, we introduce Envelope Learning, a novel method for designing Tone Transfer architectures that map musical events using a training objective at the synthesis parameter level. Our technique can render note beginnings and endings accurately and for a variety of sounds; these are essential steps for improving musical articulation, phrasing, and sound diversity with Tone Transfer. Finally, we implement a VST plugin for real-time live use and discuss possibilities for improvement. CCS CONCEPTS• Applied computing → Sound and music computing; • Computing methodologies → Neural networks; • Human-centered computing → Interaction techniques.

show abstract

Tool or Actor? Expert Improvisers' Evaluation of a Musical AI “Toddler”

Erdem,

Wallace,

Glette

et al. 2022

Computer Music Journal

View full text Add to dashboard Cite

In this article we introduce the coadaptive audiovisual instrument CAVI. This instrument uses deep learning to generate control signals based on muscle and motion data of a performer's actions. The generated signals control time-based live sound-processing modules. How does a performer perceive such an instrument? Does it feel like a machine learning-based musical tool? Or is it an actor with the potential to become a musical partner? We report on an evaluation of CAVI after it had been used in two public performances. The evaluation is based on interviews with the performers, audience questionnaires, and the creator's self-analysis. Our findings suggest that the perception of CAVI as a tool or actor correlates with the performer's sense of agency. The perceived agency changes throughout a performance based on several factors, including perceived musical coordination, the balance between surprise and familiarity, a “common sense,” and the physical characteristics of the performance setting.

show abstract

AI-terity 2.0: An Autonomous NIME Featuring GANSpaceSynth Deep Learning Model

Cited by 4 publications

References 11 publications

Embodied exploration of deep latent spaces in interactive dance-music performance

Embodied exploration of deep latent spaces in interactive dance-music performance

FM Tone Transfer with Envelope Learning

Tool or Actor? Expert Improvisers' Evaluation of a Musical AI “Toddler”

Contact Info

Product

Resources

About