Voks: Digital instruments for chironomic control of voice samples

Abstract

This paper presents Voks, a new family of digital instruments that allow for real-time control and modification of pre-recorded voice signal samples. An instrument based on Voks is made of Voks itself, the synthesis software and a given set of chironomic (hand-driven) interfaces. Rhythm can be accurately controlled thanks to a new methodology, based on syllabic control points. Timing can also be controlled with other methods, including scrubbing and playback speed variation. Pitch, vocal effort, voice tension, apparent vocal tract size, voicing ratio, aperiodicity ratio of the voice samples can be modified thanks to a real-time high-quality vocoder. Different forms of chironomic control of the vocal parameters are proposed. Pitch is controlled by continuous hand motions using a stylus on a surface (C-Voks) or a theremin (T-Voks). Other interfaces can be used as well. Syllabic rhythm is controlled using a biphasic button. Scrubbing, playback speed and timbre related parameters can be controlled using the theremin, control surfaces or continuous controllers like faders. In addition to realistic imitation of speaking or singing voices, other playing modes yield new interesting sounds. Voks participated in comparative perceptual evaluation of singing synthesis systems. It has been demonstrated in a live musical settings, using different control interfaces. In addition to musical or poetic performances, applications of performative vocal synthesis to language learning and speech reeducation are foreseen.

Prosodic Disambiguation Using Chironomic Stylization of Intonation with Native and Non-Native Speakers

“…For the present study, all syllables appeared with equal width in the interface, as French is often said to be "isochronic" [13]. Different rhythms can be realized by changing the speed of the finger's movement across the control surface (the "scrub" mode in Voks [7]). Although only syllable-level segmentation is displayed, intra-syllabic temporal modifications are possible based on the speed of finger movement within each syllabic region.…”

“…The architecture is based on Voks [7], a high-quality performative vocal synthesizer that enables the real-time melodic and rhythmic control of previously recorded or Text-to-Speech speech samples, through the use of hand gestures. It is a MaxMSP application based on the WORLD vocoder [8,9] initially developed with singing synthesis applications in mind, using a stylus on a graphic tablet or a Theremin [10,11].…”

Une nouvelle organologie de la voix : chironomie et prosodie de la parole et du chant

“…Pour parler, synthétiser n'importe quel énoncé, une approche radicalement différente de la synthèse est nécessaire. Une seconde famille d'instruments vocaux a abouti à Voks (Delalez et d'Alessandro, 2017, Locqueville et al, 2020, synthèse vocale performative par échantillonnage, dont le schéma de principe est porté Figure 2. Avant de pouvoir jouer un texte, il faut acquérir les échantillons sonores correspondant au contenu segmental (par enregistrement, synthèse à partir du texte, exploration de base de données) et les enrichir de points de contrôle rythmique.…”

“…Figure 2 : Schéma de principe de Voks. En haut le contrôle gestuel et l'interface utilisateur ; au centre le synthétiseur par échantillonnage ; à gauche la base de données d'échantillons étiquetés (Locqueville et al 2020) Pour le contrôle prosodique du rythme de parole il faut repérer des unités rythmiques, ici les syllabes, à l'aide de points de contrôle. Des marques sont apposées sur tous les noyaux syllabiques (ainsi que sur les silences suprasegmentaux) et des contre-marques sont apposée entre les noyaux vocaliques, dans les groupes consonantiques ou silences formés par les attaques et codas des syllabes).…”