Improving Reverberant Speech Separation with Synthetic Room Impulse Responses

Aralikatti, Rohith; Ratnarajah, Anton; Tang, Zhenyu; Manocha, Dinesh

doi:10.1109/asru51503.2021.9688203

Cited by 7 publications

(3 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…IRs are used in a wide range of practical applications such as audiovisual navigation [Chen et al , 2021Luo et al 2022], acoustic matching [Chen et al 2022], sound rendering [Schissler et al 2014], sound source localization [Grumiaux et al 2021], speech enhancement [Neo et al 2020], speech recognition [Malik et al 2021;Shao et al 2021], and speech separation [Aralikatti et al 2021;Jenrungrot et al 2020]. In most applications, learning-based networks are trained on large, diverse synthetic datasets and tested in real-world environments [Tang et al 2020].…”

Section: Audio Processing Using Irmentioning

confidence: 99%

MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenes

Ratnarajah,

Tang,

Aralikatti

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We propose a mesh-based neural network (MESH2IR) to generate acoustic impulse responses (IRs) for indoor 3D scenes represented using a mesh. The IRs are used to create a high-quality sound experience in interactive applications and audio processing. Our method can handle input triangular meshes with arbitrary topologies (2K -3M triangles). We present a novel training technique to train MESH2IR using energy decay relief and highlight its benefits. We also show that training MESH2IR on IRs preprocessed using our proposed technique significantly improves the accuracy of IR generation. We reduce the non-linearity in the mesh space by transforming 3D scene meshes to latent space using a graph convolution network. Our MESH2IR is more than 200 times faster than a geometric acoustic algorithm on a CPU and can generate more than 10,000 IRs per second on an NVIDIA GeForce RTX 2080 Ti GPU for a given furnished indoor 3D scene. The acoustic metrics are used to characterize the acoustic environment. We show that the acoustic metrics of the IRs predicted from our MESH2IR match the ground truth with less than 10% error. We also highlight the benefits of MESH2IR on audio and speech processing applications such as speech dereverberation and speech separation. To the best of our knowledge, ours is the first neural-network-based approach to predict IRs from a given 3D scene mesh in real-time.

show abstract

Section: Audio Processing Using Irmentioning

confidence: 99%

MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenes

Ratnarajah,

Tang,

Aralikatti

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Speech Processing using Deep Learning: IRs consist of many clues related to reproducing or understanding intelligible human speech. Synthetic datasets of IRs have been used in machine learning methods for automatic speech recognition [Ko et al 2017;Malik et al 2021;Tang et al 2020b], sound source separation [Aralikatti et al 2021;Jenrungrot et al 2020] and sound source localization [Grumiaux et al 2021]. 4.…”

Section: Introductionmentioning

confidence: 99%

GWA: A Large High-Quality Acoustic Dataset for Audio Processing

Tang,

Aralikatti,

Ratnarajah

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Figure 1: Our IR data generation pipeline starts from a 3D model of a complex scene and its visual material annotations (unstructured texts). We sample multiple collision-free source and receiver locations in the scene. We use a novel scheme to automatically assign acoustic material parameters by semantic matching from a large acoustic database. Our hybrid acoustic simulator generates accurate impulse responses (IRs), which become part of the large synthetic impulse response dataset after post-processing.

show abstract

“…are characterized by a transfer function known as the room impulse response (RIR). Measuring or simulating accurate RIRs for a given acoustic environment is vital for a variety of speech and audio applications including speech recognition [1,2], speech dereverberation [3,4], speech separation [5,6] and augmented/virtual reality [7].…”

Section: Introductionmentioning

confidence: 99%

IR-GAN: Room Impulse Response Generator for Far-Field Speech Recognition

Ratnarajah¹,

Tang

Manocha

2021

Interspeech 2021

View full text Add to dashboard Cite

We present a Generative Adversarial Network (GAN) based room impulse response generator for generating realistic synthetic room impulse responses. Our proposed generator can create synthetic room impulse responses by parametrically controlling the acoustic features captured in real-world room impulse responses. Our GAN-based room impulse response generator (IR-GAN) is capable of improving far-field automatic speech recognition in environments not known during training. We create far-field speech training set by augmenting our synthesized room impulse responses with clean Lib-riSpeech dataset [1]. We evaluate the quality of our room impulse responses on the real-world LibriSpeech test set created using real impulse responses from BUT ReverbDB [2] and AIR [3] datasets. Furthermore, we combine our synthetic data with synthetic impulse responses generated using acoustic simulators, and this combination can reduce the word error rate by up to 14.3% in far-field speech recognition benchmarks.

show abstract

Improving Reverberant Speech Separation with Synthetic Room Impulse Responses

Cited by 7 publications

References 30 publications

MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenes

MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenes

GWA: A Large High-Quality Acoustic Dataset for Audio Processing

IR-GAN: Room Impulse Response Generator for Far-Field Speech Recognition

Contact Info

Product

Resources

About