Example-guided physically based modal sound synthesis

Abstract

Linear modal synthesis methods have often been used to generate sounds for rigid bodies. One of the key challenges in widely adopting such techniques is the lack of automatic determination of satisfactory material parameters that recreate realistic audio quality of sounding materials. We introduce a novel method using pre-recorded audio clips to estimate material parameters that capture the inherent quality of recorded sounding materials. Our method extracts perceptually salient features from audio examples. Based on psychoacoustic principles, we design a parameter estimation algorithm using an optimization framework and these salient features to guide the search of the best material parameters for modal synthesis. We also present a method that compensates for the differences between the realworld recording and sound synthesized using solely linear modal synthesis models to create the final synthesized audio. The resulting audio generated from this sound synthesis pipeline well preserves the same sense of material as a recorded audio example. Moreover, both the estimated material parameters and the residual compensation naturally transfer to virtual objects of different sizes and shapes, while the synthesized sounds vary accordingly. A perceptual study shows the results of this system compares well with real-world recordings in terms of material perception.

Digital Synthesis of Sound Generated by Tibetan Bowls and Bells

“…Both cases entailed a steady state analysis described in Eq. (25). The forcing in the system originated from a harmonic force applied to an element node.…”

“…Historically, the first method was the use of lumped mass-spring networks (Cadoz et al, 1983), in which connected networks with lumped parameters (mass and spring) describe, e.g., string vibration in 1D and membrane vibration in 2D. Another method used a modal synthesis in which the natural frequencies and mode shapes are both obtained for the differential equations describing the system, followed by receiving the system response by use of modal superposition (Ren et al, 2013). Another important synthesis method is the digital waveguides method as presented in a very computationally effective way by Julius Smith (Smith, 1992).…”

Computational design of metallophone contact sounds

“…The example-based synthesis work of Ren et al [2013] matches an object's sound spectrum to a recorded sound using modal analysis and a Nelder-Mead optimizer. This transfers recorded sounds to a virtual object by finding optimal material parameters.…”

“…We exploit fast physicsbased modal sound simulation from computer animation [van den Doel and Pai 1998;O'Brien et al 2002;Raghuvanshi and Lin 2006;James et al 2006;Chadwick et al 2009;Zheng and James 2010;Zheng and James 2011;Lloyd et al 2011;Ren et al 2013] and acoustics [Chaigne and Doutaut 1997]. These rely on modal analysis [De Poli et al 1991] ( §4.1), which is widely used in computational mechanics.…”

Tabletop Ensemble

“…More importantly, parameters like proportional damping coefficients do not directly map to real-world materials, therefore impossible to look up for modal analysis. In our preprocessing, we adopt a new example-guided parameter estimation algorithm, using only a single example audio recording for each material [Ren et al 2011]. Guided by a sample audio recording of a xylophone bar and of a drum, this automatic, offline process facilitates quick determination of material parameters of realistic sounding materials the simulated xylophone and a drum set.…”