Board # 150 : MAKER: Painting Pitches

Abstract

The following paper presents a system for creating real-time visuals based on multiple simultaneous vocal inputs. The goal of this work is to augment live musical performances by visually conveying the spirit and structure of the piece. Using the Processing programming language, sounds are analyzed and return pitch and amplitude as numerical values. Visuals representing pitch and amplitude for each of the musicians are created in real-time and are projected concurrently with the live musical performance. This process is demonstrated by work presented here with a professional 8-voice ensemble. As part of this project, a library of functions is being created and shared to allow others to implement similar productions. IntroductionThe initial goal of this work is to create a visual counterpart to the on-tour performances of Cantus, an a cappella men's vocal ensemble. The artistic goal of the work is to explore the connections between sight and sound and to add musically relevant and engaging visuals that would react to the singers' voices in real time. A secondary goal of the efforts is to create these functionalities in a way that can easily be modified and shared by other users.

Selective listening attention enhancement, using a simultaneous visual and haptic stimuli

“…In addition to these examples, there are other studies that synchronized music with a haptic stimulus (Armitage & Ng, 2016;Merchel, Altinsoy, & Stamm, 2010), a visual stimulus (Meuer et al, 2017), or a combination of both (Rodet et al, 2005;Tanabe et al, 2016). Merchel et al and Armitage and Ng designed specific hardware solutions to improve participants playing performance.…”