HRTF Representation with Convolutional Auto-encoder

Chen, Wei; Hu, Ruimin; Wang, Xiaochen; Li, Dengshi

doi:10.1007/978-3-030-37731-1_49

Cited by 2 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In previous works [27], [28], individual HRTF synthesis with individual pinna images employed three sub-networks: the variational autoencoder (VAE), fully connected (FC) layers, and conditional VAE (CVAE). While VAE and CVAE models are commonly used for data reconstruction [29], multi-step learning methods can be inefficient and yield suboptimal optimization results since each network is trained separately [48]. To address this limitation, we propose an end-to-end network for unified optimization of HRTF individualization, spanning from the pinna image to HRTF magnitude.…”

Section: Neural Network Structure and Direction-wise Trainingmentioning

confidence: 99%

“…More recently, based on experimental findings highlighting the role of ear pinna in generating spectral cues of HRTFs [26], DNN architectures that generate the magnitude spectra of individual HRTFs using only pinna images have been proposed [27], [28]. These DNNs typically employ an autoencoder structure, known for efficient dimensionality reduction of HRTFs [29]. They consist of three sub-networks that convert pinna images through latent variables to HRTF magnitude.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PRTFNet: HRTF Individualization for Accurate Spectral Cues Using a Compact PRTF

Ko,

Lee,

Nam

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Spatial audio rendering relies on accurate localization perception, which requires individual head-related transfer functions (HRTFs). Previous methods based on deep neural networks (DNNs) for predicting HRTF magnitude spectra from pinna images used HRTF log-magnitude as the network output during the training stage. However, HRTFs encompass the acoustical characteristics of the head and torso, making it challenging to reconstruct the spectral cues necessary for elevation localization. To tackle this issue, we propose PRTFNet to reconstruct the individual spectral cues in HRTFs by mitigating the influence of the head and torso. PRTFNet consists of an end-to-end convolutional neural network (CNN) model and leverages a compact pinna-related transfer function (PRTF) that eliminates the impact of sound reflections from the head and torso in the head-related impulse response (HRIR) as network output. Additionally, we introduce HRTF phase personalization, a technique that utilizes the phase spectra of a selected HRTFs from a database and adjusts the phase by multiplying it by the ratio of the target listener's head width to that of the subject of the selected HRTFs. We evaluated the proposed HRTF individualization methods using the HUTUBS dataset, and the results demonstrate that PRTFNet is highly effective in reconstructing the first and second spectral cues. In terms of log spectral distortion (LSD) and effective LSD (LSDE), PRTFNet outperforms previous deep learning-based model. Furthermore, multiplying the selected phase by the head width ratio reduces the root mean square error (RMSE) of interaural time difference (ITD) by 0.003 ms.INDEX TERMS Head-related transfer functions, individualization, pinna-related transfer functions, spectral cues, spatial hearing.

show abstract

Section: Neural Network Structure and Direction-wise Trainingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

PRTFNet: HRTF Individualization for Accurate Spectral Cues Using a Compact PRTF

Ko,

Lee,

Nam

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…A separate DNN is then trained to predict these latent vectors using anthropometric data as input. Lastly, in a very recent paper, W. Chen et al [6] train a convolutional denoising autoencoder on 2D frequency-elevation input features derived from listener-specific directional components of HRTFs, with the purpose of optimizing HRTF dataset storage.…”

Section: Related Workmentioning

confidence: 99%

A hybrid approach to structural modeling of individualized HRTFs

Miccini

Spagnol

2021

2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)

View full text Add to dashboard Cite

We present a hybrid approach to individualized head-related transfer function (HRTF) modeling which requires only 3 anthropometric measurements and an image of the pinna. A prediction algorithm based on variational autoencoders synthesizes a pinna-related response from the image, which is used to filter a measured head-andtorso response. The interaural time difference is then manipulated to match that of the HUTUBS dataset subject minimizing the predicted localization error. The results are evaluated using spectral distortion and an auditory localization model. While the latter is inconclusive regarding the efficacy of the structural model, the former metric shows promising results with encoding HRTFs.

show abstract

HRTF Representation with Convolutional Auto-encoder

Cited by 2 publications

References 16 publications

PRTFNet: HRTF Individualization for Accurate Spectral Cues Using a Compact PRTF

PRTFNet: HRTF Individualization for Accurate Spectral Cues Using a Compact PRTF

A hybrid approach to structural modeling of individualized HRTFs

Contact Info

Product

Resources

About