A hybrid model of timbre perception.

Abstract

Timbre is a fundamental attribute of sound. It is important in differentiating between musical sounds, speech utterances, and characterizing everyday sounds in our environment as well as novel synthetic sounds. A hybrid model of timbre perception, which integrates the concepts of color and texture of sound, is proposed. The color of sound is described in terms of an instantaneous (or ideally timeless) spectral envelope, while the texture of a sound describes the temporal structure of the sound, as the sequential changes of color with an arbitrary range of time-scale. The computational implementation of this model represents a sound’s color as the spectral envelope of a specific window, and its texture as the granularity (or microtexture) of the corresponding window. The temporal structures across windows from both color and texture parts of the model serve as the texture of a sound in a larger time-scale. In support of the proposed theory a series of psychoacoutic experiments was performed. The quantitative relationship between the spectral envelope and subjective perception of complex tones used Mel-frequency cepstral coefficients as a representation. A perceptually tested quantitative representation of texture was established using normalized echo density.

Usage of adaptive features in sound synthesis GUIs

“…Its complexity lies in its multidimensionality [5], which hasn't been reduced to a few perceptive variables. Some scholars have tried to solve this issue by introducing the concept of timbre space [6] [7] [8], conducting experiments using levels of perceived similarity among various pairs of sounds attempting to find lowdimensional timbre spaces significant to human perception, analogous to the RGB scale for color. The hypothesis is that these variables, which are significant for the ear, will be the variables of future synthesizers.…”

Many-to-Many Voice Conversion Based Feature Disentanglement Using Variational Autoencoder

“…Voice conversion task is a technique which is used to transform a speaker's identity from the source speaker to the target speaker without changing the linguistic content [1]. The identity of a speaker, which depends on the voice timbre of a speaker, can be extracted from Mel-frequency cepstral coefficient (MFCC) by using statistical approaches [2]. These extracted features are then switched between source speakers and target speakers to perform VC.…”

Composing with Multidimensional Timbre Representations

“…As timbre became increasingly central in my composition, I adopted a hybrid model that integrates both the "color" and "texture" of sound and incorporates both static and dynamic attributes of timbre. The "color" of sound is described in terms of an "instantaneous snapshot of the spectral envelope, " while the "texture" of a sound describes the "sequential changes in color with an arbitrary time scale" [2].Hiroko Terasawa and Jonathan Berger developed this model of timbre at Stanford University's Center for Computer Research in Music and Acoustics (CCRMA). This view hints at two major compositional elements in a piece:(1) static, vertical pitch and chordal structures and (2) dynamic, horizontal temporal processes.…”

“…As timbre became increasingly central in my composition, I adopted a hybrid model that integrates both the "color" and "texture" of sound and incorporates both static and dynamic attributes of timbre. The "color" of sound is described in terms of an "instantaneous snapshot of the spectral envelope, " while the "texture" of a sound describes the "sequential changes in color with an arbitrary time scale" [2].…”