Sliced Wasserstein Distance for Learning Gaussian Mixture Models

Kolouri, Soheil; Rohde, Gustavo K.; Hoffmann, Herbert

doi:10.1109/cvpr.2018.00361

Cited by 99 publications

(109 citation statements)

References 47 publications

Supporting

Mentioning

109

Contrasting

Order By: Relevance

“…For this reason, efficient approximations and variants for it have been an active research area. In this paper, we used the Sliced Wasserstein Distance (SWD) [19], which is a good approximate of optimal transport [20] and additionally can be computed more efficiently.…”

Section: Proposed Solutionmentioning

confidence: 99%

“…Second, SWD is non-zero even for two probability distributions with non-overlapping supports. As a result, it has non-vanishing gradients, and first-order gradient-based optimization algorithms can be used to solve optimization problems involving SWD terms [16,20]. This is important, as most optimization problems for training deep neural networks are solved using gradient-based methods, e.g., Stochastic Gradient Descent (SGD).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Deep Transfer Learning for Few-Shot SAR Image Classification

et al. 2019

Self Cite

View full text Add to dashboard Cite

The reemergence of Deep Neural Networks (DNNs) has lead to high-performance supervised learning algorithms for the Electro-Optical (EO) domain classification and detection problems. This success is because generating huge labeled datasets has become possible using modern crowdsourcing labeling platforms such as Amazon’s Mechanical Turk that recruit ordinary people to label data. Unlike the EO domain, labeling the Synthetic Aperture Radar (SAR) domain data can be much more challenging, and for various reasons, using crowdsourcing platforms is not feasible for labeling the SAR domain data. As a result, training deep networks using supervised learning is more challenging in the SAR domain. In the paper, we present a new framework to train a deep neural network for classifying Synthetic Aperture Radar (SAR) images by eliminating the need for a huge labeled dataset. Our idea is based on transferring knowledge from a related EO domain problem, where labeled data are easy to obtain. We transfer knowledge from the EO domain through learning a shared invariant cross-domain embedding space that is also discriminative for classification. To this end, we train two deep encoders that are coupled through their last year to map data points from the EO and the SAR domains to the shared embedding space such that the distance between the distributions of the two domains is minimized in the latent embedding space. We use the Sliced Wasserstein Distance (SWD) to measure and minimize the distance between these two distributions and use a limited number of SAR label data points to match the distributions class-conditionally. As a result of this training procedure, a classifier trained from the embedding space to the label space using mostly the EO data would generalize well on the SAR domain. We provide a theoretical analysis to demonstrate why our approach is effective and validate our algorithm on the problem of ship classification in the SAR domain by comparing against several other competing learning approaches.

show abstract

Section: Proposed Solutionmentioning

confidence: 99%

mentioning

confidence: 99%

Deep Transfer Learning for Few-Shot SAR Image Classification

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…It can be shown that training with the MLE criteria converges to a minimization of the KL-divergence as the sample size increases [23]. From Eqn (1) we see that any model admitting a differentiable density p θ (x) can be trained via backpropagation.…”

Section: A Maximum Likelihood Trainingmentioning

confidence: 99%

Deep Generative Modeling of LiDAR Data

Caccia

Hoof

Courville

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Building models capable of generating structured output is a key challenge for AI and robotics. While generative models have been explored on many types of data, little work has been done on synthesizing lidar scans, which play a key role in robot mapping and localization. In this work, we show that one can adapt deep generative models for this task by unravelling lidar scans into a 2D point map. Our approach can generate high quality samples, while simultaneously learning a meaningful latent representation of the data. We demonstrate significant improvements against state-of-the-art point cloud generation methods. Furthermore, we propose a novel data representation that augments the 2D signal with absolute positional information. We show that this helps robustness to noisy and imputed input; the learned model can recover the underlying lidar scan from seemingly uninformative data.

show abstract

“…The gradients on this space are also well behavioured, which promises to achieve superior optimisation. Most recently, Kolouri et al (Kolouri, Rohde, and Hoffmann 2018) have adopted the Wasserstein distance for solving GMM problems. However, the aforementioned probability constraint enforces an extremely small stepsize (learning rate) during optimisation.…”

Section: Introductionmentioning

confidence: 99%

Solving General Elliptical Mixture Models through an Approximate Wasserstein Manifold

Xiang

et al. 2020

AAAI

View full text Add to dashboard Cite

We address the estimation problem for general finite mixture models, with a particular focus on the elliptical mixture models (EMMs). Compared to the widely adopted Kullback–Leibler divergence, we show that the Wasserstein distance provides a more desirable optimisation space. We thus provide a stable solution to the EMMs that is both robust to initialisations and reaches a superior optimum by adaptively optimising along a manifold of an approximate Wasserstein distance. To this end, we first provide a unifying account of computable and identifiable EMMs, which serves as a basis to rigorously address the underpinning optimisation problem. Due to a probability constraint, solving this problem is extremely cumbersome and unstable, especially under the Wasserstein distance. To relieve this issue, we introduce an efficient optimisation method on a statistical manifold defined under an approximate Wasserstein distance, which allows for explicit metrics and computable operations, thus significantly stabilising and improving the EMM estimation. We further propose an adaptive method to accelerate the convergence. Experimental results demonstrate the excellent performance of the proposed EMM solver.

show abstract

Sliced Wasserstein Distance for Learning Gaussian Mixture Models

Cited by 99 publications

References 47 publications

Deep Transfer Learning for Few-Shot SAR Image Classification

Deep Transfer Learning for Few-Shot SAR Image Classification

Deep Generative Modeling of LiDAR Data

Solving General Elliptical Mixture Models through an Approximate Wasserstein Manifold

Contact Info

Product

Resources

About