Auditory-visual scenes for hearing research

Abstract

While experimentation with synthetic stimuli in abstracted listening situations has a long standing and successful history in hearing research, an increased interest exists on closing the remaining gap towards real-life listening by replicating situations with high ecological validity in the lab. This is important for understanding the underlying auditory mechanisms and their relevance in real-life situations as well as for developing and evaluating increasingly sophisticated algorithms for hearing assistance. A range of ‘classical’ stimuli and paradigms have evolved to de-facto standards in psychoacoustics, which are simplistic and can be easily reproduced across laboratories. While they ideally allow for across laboratory comparisons and reproducible research, they, however, lack the acoustic stimulus complexity and the availability of visual information as observed in everyday life communication and listening situations. This contribution aims to provide and establish an extendable set of complex auditory-visual scenes for hearing research that allow for ecologically valid testing in realistic scenes while also supporting reproducibility and comparability of scientific results. Three virtual environments are provided (underground station, pub, living room), consisting of a detailed visual model, an acoustic geometry model with acoustic surface properties as well as a set of acoustic measurements in the respective real-world environments. The current data set enables i) audio–visual research in a reproducible set of environments, ii) comparison of room acoustic simulation methods with “ground truth” acoustic measurements, iii) a condensation point for future extensions and contributions for developments towards standardized test cases for ecologically valid hearing research in complex scenes.

Development of Virtual Reality scenes for clinical use with hearing device fine-tuning

“…In this work, we described what steps need to be taken to implement VR scenes for hearing device fine-tuning. It shows that considerable effort was needed to further develop the virtual environments as they were published in [8] to VR scenes that can be used for hearing device finetuning in the clinic. We therefore decided to publish the VR scenes as an application that is ready-to-use.…”

“…The virtual pub is based on the pub presented in [8,17]. Diffuse babble noise fitting this environment was generated by calculating the sum of the convolutions of the recorded IRs from the pub [17] with conversation snippets.…”

“…Visual cues can also guide spatial attention [5] and affect self-motion [6,7]. While a previous study has published an extendable set of complex audiovisual environments for hearing research [8], we found considerable effort was still needed to turn them into VR scenes applicable for hearing device fine-tuning in the clinic. The aim of this work is to give insight into the choices that need to be made in the implementation of VR scenes.…”

“…Section 4 describes the technical evaluation that was carried out with two hearing aid brands to determine whether the automatic scene classifiers classified the VR scenes as the intended category. The VR scenes developed in this work are published [9] as an application that is ready-to-use but less modifiable than the set of audiovisual environments in [8].…”

A model framework for simulating spatial hearing of bilateral cochlear implant users

“…Although it was only tested with lateralization and localization experiments (see also Supplementary materials) in this study, by selecting different stage combinations and different decision models, its applications can go beyond the provided experimental examples. By referring to the MATLAB examples provided, users are able to replace the current functions with customized ones, such as incorporating other HRIRs or BRIRs for complex acoustic environments (e.g., [53,[92][93][94][95][96][97][98]), self-developed AGCs, different coding strategies, different AN models, or different decision models. Each model stage alone can be used or extended for both research and education purposes.…”