Wasserstein Measure Coresets

Genevay, Aude; Solomon, Justin

doi:10.48550/arxiv.1805.07412

Cited by 5 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, this problem formation ignores the fact that a dataset is an empirical sample of data distribution, which describes a learning task. To build the bridge between coreset and Wasserstein distance, we introduce the idea of Wasserstein coreset (Claici, Genevay, and Solomon 2020) and follow the notations and definitions of coreset and measure coreset. Then we have the following proposition:…”

Section: Methodsmentioning

confidence: 99%

An Optimal Transport Approach to Deep Metric Learning (Student Abstract)

Dou

Luo

Yang

2022

AAAI

View full text Add to dashboard Cite

Capturing visual similarity among images is the core of many computer vision and pattern recognition tasks. This problem can be formulated in such a paradigm called metric learning. Most research in the area has been mainly focusing on improving the loss functions and similarity measures. However, due to the ignoring of geometric structure, existing methods often lead to sub-optimal results. Thus, several recent research methods took advantage of Wasserstein distance between batches of samples to characterize the spacial geometry. Although these approaches can achieve enhanced performance, the aggregation over batches definitely hinders Wasserstein distance's superior measure capability and leads to high computational complexity. To address this limitation, we propose a novel Deep Wasserstein Metric Learning framework, which employs Wasserstein distance to precisely capture the relationship among various images under ranking-based loss functions such as contrastive loss and triplet loss. Our method directly computes the distance between images, considering the geometry at a finer granularity than batch level. Furthermore, we introduce a new efficient algorithm using Sinkhorn approximation and Wasserstein measure coreset. The experimental results demonstrate the improvements of our framework over various baselines in different applications and benchmark datasets.

show abstract

Section: Methodsmentioning

confidence: 99%

An Optimal Transport Approach to Deep Metric Learning (Student Abstract)

Dou

Luo

Yang

2022

AAAI

View full text Add to dashboard Cite

show abstract

“…Sensitivity-based methods, such as the k-clustering problems take into account the importance of samples by approximate probability (Bachem, Lucic, and Lattanzi 2018;Bateni et al 2014). Distributionbased methods typically require consideration of the underlying data distribution, such as designing the coreset based on Reproducing Kernel Hilbert Space (RKHS) theory (Chen, Welling, and Smola 2012) or utilizing the integral probability metric in the context of optimal transport theory (Claici, Genevay, and Solomon 2018). However, these traditional coreset methods (Feldman, Faulkner, and Krause 2011;Bachem, Lucic, and Krause 2015;Zhang et al 2023) face challenges due to their high computational complexity and reliance on fixed data representations (which are seldom suited for image data).…”

Section: Related Workmentioning

confidence: 99%

Feature Distribution Matching by Optimal Transport for Effective and Robust Coreset Selection

Xiao,

Chen,

Shan

et al. 2024

AAAI

View full text Add to dashboard Cite

Training neural networks with good generalization requires large computational costs in many deep learning methods due to large-scale datasets and over-parameterized models. Despite the emergence of a number of coreset selection methods to reduce the computational costs, the problem of coreset distribution bias, i.e., the skewed distribution between the coreset and the entire dataset, has not been well studied. In this paper, we find that the closer the feature distribution of the coreset is to that of the entire dataset, the better the generalization performance of the coreset, particularly under extreme pruning. This motivates us to propose a simple yet effective method for coreset selection to alleviate the distribution bias between the coreset and the entire dataset, called feature distribution matching (FDMat). Unlike gradient-based methods, which selects samples with larger gradient values or approximates gradient values of the entire dataset, FDMat aims to select coreset that is closest to feature distribution of the entire dataset. Specifically, FDMat transfers coreset selection as an optimal transport problem from the coreset to the entire dataset in feature embedding spaces. Moreover, our method shows strong robustness due to the removal of samples far from the distribution, especially for the entire dataset containing noisy and class-imbalanced samples. Extensive experiments on multiple benchmarks show that FDMat can improve the performance of coreset selection than existing coreset methods. The code is available at https://github.com/successhaha/FDMat.

show abstract

“…Constructing a core-set from a large dataset is an optimization problem of finding a smaller set that can best approximate the original dataset with respect to a certain measure. Claici et al [8] leveraged optimal transport theory and introduced Wasserstein measure to calculate the core-set. Their work aims to minimize the Wasserstein distance of the core-set from a given input data distribution.…”

Section: Problem 3: Transport-based Coresetmentioning

confidence: 99%

Teaching Networks to Solve Optimization Problems

Liu¹,

Lu²,

Abbasi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Leveraging machine learning to optimize the optimization process is an emerging field which holds the promise to bypass the fundamental computational bottleneck caused by traditional iterative solvers in critical applications requiring near-real-time optimization. The majority of existing approaches focus on learning data-driven optimizers that lead to fewer iterations in solving an optimization. In this paper, we take a different approach and propose to replace the iterative solvers altogether with a trainable parametric set function that outputs the optimal arguments/parameters of an optimization problem in a single feed-forward. We denote our method as, Learning to Optimize the Optimization Process (LOOP). We show the feasibility of learning such parametric (set) functions to solve various classic optimization problems, including linear/nonlinear regression, principal component analysis, transport-based core-set, and quadratic programming in supply management applications. In addition, we propose two alternative approaches for learning such parametric functions, with and without a solver in the-LOOP. Finally, we demonstrate the effectiveness of our proposed approach through various numerical experiments.

show abstract

Wasserstein Measure Coresets

Cited by 5 publications

References 0 publications

An Optimal Transport Approach to Deep Metric Learning (Student Abstract)

An Optimal Transport Approach to Deep Metric Learning (Student Abstract)

Feature Distribution Matching by Optimal Transport for Effective and Robust Coreset Selection

Teaching Networks to Solve Optimization Problems

Contact Info

Product

Resources

About