Local conformal autoencoder for standardized data coordinates

Peterfreund, Erez; Lindenbaum, Ofir; Dietrich, Felix; Bertalan, Tom; Gavish, Matan; Kevrekidis, Ioannis G.; Coifman, Ronald R.

doi:10.1073/pnas.2014627117

Cited by 13 publications

(10 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach is a form of manifold learning [22], [28], [29]. Manifold learning has recently been proposed for semisupervised source localization in room acoustics [15].…”

Section: Introductionmentioning

confidence: 99%

“…We present our VAE-SSL method as an alternative to this manifold learning approach. Recent work has indicated the capabilities of deep learning in manifold learning [28]. VAE-SSL uses the non-linear modeling capabilities of deep generative modeling to obtain an SSL localization approach which is less reliant on preprocessing and hand-engineering of latent representations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Semi-Supervised Source Localization in Reverberant Environments With Deep Generative Modeling

et al. 2021

View full text Add to dashboard Cite

We propose a semi-supervised approach to acoustic source localization in reverberant environments based on deep generative modeling. Localization in reverberant environments remains an open challenge. Even with large data volumes, the number of labels available for supervised learning in reverberant environments is usually small. We address this issue by performing semi-supervised learning (SSL) with convolutional variational autoencoders (VAEs) on reverberant speech signals recorded with microphone arrays. The VAE is trained to generate the phase of relative transfer functions (RTFs) between microphones, in parallel with a direction of arrival (DOA) classifier based on RTF-phase. These models are trained using both labeled and unlabeled RTF-phase sequences. In learning to perform these tasks, the VAE-SSL explicitly learns to separate the physical causes of the RTF-phase (i.e., source location) from distracting signal characteristics such as noise and speech activity. Relative to existing semi-supervised localization methods in acoustics, VAE-SSL is effectively an end-to-end processing approach which relies on minimal preprocessing of RTF-phase features. As far as we are aware, our paper presents the first approach to modeling the physics of acoustic propagation using deep generative modeling. The VAE-SSL approach is compared with two signal processing-based approaches, steered response power with phase transform (SRP-PHAT) and MUltiple SIgnal Classification (MUSIC), as well as fully supervised CNNs. We find that VAE-SSL can outperform the conventional approaches and the CNN in label-limited scenarios. Further, the trained VAE-SSL system can generate new RTF-phase samples, which shows the VAE-SSL approach learns the physics of the acoustic environment. The generative modeling in VAE-SSL thus provides a means of interpreting the learned representations.

show abstract

“…This approach is a form of manifold learning [22], [28], [29]. Manifold learning has recently been proposed for semisupervised source localization in room acoustics [15].…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Semi-Supervised Source Localization in Reverberant Environments With Deep Generative Modeling

et al. 2021

View full text Add to dashboard Cite

show abstract

“…This approach is a form of manifold learning [19], [26], [27]. Manifold learning has recently been proposed for semisupervised source localization in room acoustics [14].…”

Section: Introductionmentioning

confidence: 99%

“…We present our VAE-SSL method as an alternative to this manifold learning approach. Recent work has indicated the capabilities of deep learning in manifold learning [26]. VAE-SSL uses the non-linear modeling capabilities of deep generative modeling to obtain a semi-supervised localization approach which is less reliant on preprocessing and handengineering of latent representations.…”

Section: Introductionmentioning

confidence: 99%

Semi-supervised source localization in reverberant environments with deep generative modeling

Bianco¹,

Gannot²,

Fernández-Grande³

et al. 2021

Preprint

View full text Add to dashboard Cite

A semi-supervised approach to acoustic source localization in reverberant environments, based on deep generative modeling, is proposed. Localization in reverberant environments remains an open challenge. Even with large data volumes, the number of labels available for supervised learning in reverberant environments is usually small. We address this issue by performing semi-supervised learning (SSL) with convolutional variational autoencoders (VAEs) on speech signals in reverberant environments. The VAE is trained to generate the phase of relative transfer functions (RTFs) between microphones, in parallel with a direction of arrival (DOA) classifier based on RTF-phase, on both labeled and unlabeled RTF samples. In learning to perform these tasks, the VAE-SSL explicitly learns to separate the physical causes of the RTF-phase (i.e., source location) from distracting signal characteristics such as noise and speech activity. Relative to existing semi-supervised localization methods in acoustics, VAE-SSL is effectively an end-to-end processing approach which relies on minimal preprocessing of RTF-phase features. The VAE-SSL approach is compared with the steered response power with phase transform (SRP-PHAT) and fully supervised CNNs. We find that VAE-SSL can outperform both SRP-PHAT and CNN in label-limited scenarios. Further, the trained VAE-SSL system can generate new RTF-phase samples, which shows the VAE-SSL approach learns the physics of the acoustic environment. The generative modeling in VAE-SSL thus provides a means of interpreting the learned representations.

show abstract

“…However, in modern Big-data related problems it is often the case that functions have a high dimensional target. Therefore, many works concern some type of regression in high dimensional space [2,5,8,11,13,14,16]. Furthermore, in [1,3,6,15], the estimation of low dimensional manifolds embedded in very high dimensional domain is performed by means of non-parametric estimation.…”

Section: Introductionmentioning

confidence: 99%

Convergence rates of vector-valued local polynomial regression

Aizenbud¹,

Sober²

2021

Preprint

View full text Add to dashboard Cite

Non-parametric estimation of functions as well as their derivatives by means of localpolynomial regression is a subject that was studied in the literature since the late 1970's. Given a set of noisy samples of a C k smooth function, we perform a local polynomial fit, and by taking its m-th derivative we obtain an estimate for the m-th function derivative. The known optimal rates of convergence for this problem for a k-times smooth function f : R d → R are n − k−m 2k+d . However in modern applications it is often the case that we have to estimate a function operating to R D , for D ≫ d extremely large. In this work, we prove that these same rates of convergence are also achievable by local-polynomial regression in case of a high dimensional target, given some assumptions on the noise distribution. This result is an extension to Stone's seminal work from 1980 to the regime of high-dimensional target domain. In addition, we unveil a connection between the failure probability ε and the number of samples required to achieve the optimal rates.

show abstract

Local conformal autoencoder for standardized data coordinates

Cited by 13 publications

References 29 publications

Semi-Supervised Source Localization in Reverberant Environments With Deep Generative Modeling

Semi-Supervised Source Localization in Reverberant Environments With Deep Generative Modeling

Semi-supervised source localization in reverberant environments with deep generative modeling

Convergence rates of vector-valued local polynomial regression

Contact Info

Product

Resources

About