Timbre transposition based on time-varying spectral analysis of continuous monophonic audio and precomputed spectral libraries

Abstract

A sinusoidal model for solo musical sounds consisting of time-varying harmonic amplitudes and frequencies allows for convenient temporal and spectral modifications. With a harmonic model, analysis frames can be grouped by fundamental frequency (F0) and then clustered in terms of their harmonic spectra. The resulting cluster centroid spectra are used as spectral libraries. When continuous audio monophonic passages are analyzed in the form of harmonic components, F0-vs.-time data are used to guide the extraction of parameters from the sound in order to find appropriate library spectra for resynthesis. Two methods for finding appropriate spectra are: 1) best rms match with the incoming spectra and 2) best spectral centroid match. These give similar results, but centroid matching yields smoother spectra over time. Timbre transposition is performed by using a library that belongs to another instrument. We have found that when the target instrument has a unique timbral quality based on its spectrum, the synthesis sounds mostly like that instrument. However, if the target instrument's spectral characteristic is not sufficiently differentiated from the source, the source timbral quality may dominate, probably due to its temporal behavior being transmitted. Results will be demonstrated by audio examples. Acoustics 08 Paris 10733

Perceptually Correlated Parameters of Musical Instrument Tones

“…The point of this study, presented as a talk in 2008 (Beauchamp, Bay, 2008), was to explore synthesis using a small set of time-variable control parameters and a family of spectral envelopes (Beauchamp, 2007;Luce, Clark, 1967), which represent a particular instrument, but then switch the spectral envelope family to a different instrument and see what happens. Either the spectral envelopes or the temporal data will dominate, or a hybrid instrument that shares characteristics of both instruments will be produced.…”

“…Three projects are reviewed, a timbre discrimination study (McAdams et al, 1999), a timbre dissimilarity judgment study with MDS solution Hall et al, 2010), and a timbre transposition study (Beauchamp, Bay, 2008).…”