We present the Latent Timbre Synthesis (LTS), a new audio synthesis method using Deep Learning. The synthesis method allows composers and sound designers to interpolate and extrapolate between the timbre of multiple sounds using the latent space of audio frames. We provide the details of two Variational Autoencoder architectures for LTS, and compare their advantages and drawbacks. The implementation includes a fully working application with graphical user interface, called interpolate two, which enables practitioners to explore the timbre between two audio excerpts of their selection using interpolation and extrapolation in the latent space of audio frames. Our implementation is open-source, and we aim to improve the accessibility of this technology by providing a guide for users with any technical background.

No abstract

Today our cultural events are dominated by visual information based on sight and sound, but hardly at all on the combined senses of touch and sound. Visually impaired users, lacking sight are not often able to engage in cultural events. Compensated audio-description or talking books are standard products that visually impaired people can buy to imagine feature film stories or decide what is happening on the stage. Very little theatre, dance or art events exist in which these people can actually participate. Interfaces are not often designed which promote communication between impaired actors, nor allow them to navigate and control audio-visual information on the mediated or digital stage. As neuroscientists suggest, the unique cross-modal potentials of human sensory perception could be augmented by electronic devices, which in turn might communicate with sighted audiences. Our research group is interested to address these problems by constructing ergonomic HCI (Human Computer Interfaces) that can explore the above problems including research into orientation, cognition mapping and external audio-visual device control. Consequently, we are implementing our discoveries in artificial systems, which can interact intelligently with people on the digitally simulated stage.Keywords: HCI, cross-modal interaction, visually impaired users, mediated stages, ergonomic design. The e-skin approach"E-skin" is a set of interfaces, which constitute our past and present attempts to electronically simulate the perceptive modalities of the human skin: pressure, temperature, vibration and proprioception. These four modalities constitute our biggest human organ, constantly detecting and reacting to environmental realities.Our research team is engaged with the mimicry of skin, alongside extensive user testing to assure we make relevant developments for cross-modal potentials, including tactile and acoustic feedback, cognitive mapping and embodied interaction. Cross-modal interaction is the label used in neuroscience to describe how certain features of objects perceived in ones sensory modes go-together with features in another sensory mode.We are particularly interested in the relationship between tactility and acoustic feedback as well as the mediated stage as a feedback loop or "visual substitute". What is the actual relationship between pressure and sound, temperature and volume and proprioception and vibration? Our group explores cognitive mapping, touch, movement, electro-stimulation on the skin itself, gesture and relief differentiation, in order to uncover some answers to this large question. Cognitive mapping exercises or tests about the composition of acquired codes, stores, recalls and decodes, determine relative locations and attributes in the spatial environment of the stage itself.Our mediated stage is a hybrid platform using surround sound and three screens with related potentials of communication and movement and gesture as well as audio-visual response or what theorists call "embodied interaction". As Paul Dour...

No abstract

Piano&Dancer is an interactive piece for a dancer and an electromechanical acoustic piano. e piece presents the dancer and the piano as two performers on stage whose bodily movements are mutually interdependent. is interdependence reveals a close relationship between physical and musical gestures. Accordingly, the realisation of the piece has been based on creative processes that merge choreographic and compositional methods. In order to relate the expressive movement qualities of a dancer to the creation of musical material, the piece employs a variety of techniques. ese include methods for movement tracking and feature analysis, generative algorithms for creating musical structures, and the application of non-conventional scales and chord transformations to shape the modal characteristics of the music. e publication contextualises Piano&Dancer by relating its creation to concepts of embodiment, interactivity and musical structure and by discussing opportunities for creative cross-fertilisation between dance choreography and musical composition. It also provides some details about the challenges and potentials of integrating a mechanical musical instrument into an interactive se ing for a dance performance. Finally, the paper highlights some of the technical and aesthetic principles that were used in order to connect expressive qualities of body movements to the creation of music structures. CCS CONCEPTS •Human-centered computing → Interaction design theory, concepts and paradigms; Interaction design process and methods; Gestural input; •Applied computing → Performing arts; Sound and music computing;