Blind Source Separation Using Mixtures of Alpha-Stable Distributions

Keriven, Nicolas; Deleforge, Antoine

doi:10.1109/icassp.2018.8462095

Cited by 5 publications

(5 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To solve clustering and density modeling tasks in a compressive manner, previous works focused on random Fourier features (RFF), which consist in using the complex exponential as the nonlinear function [44] in the feature map. This has been applied to clustering and fitting parametric mixture models, such as Gaussian mixture models [33] or alpha-stable distributions [35].…”

Section: Definition 2 (K-means Clustering Task) Given An Integer > 0 K-means Clustering Consists In Finding Centroidsmentioning

confidence: 99%

Compressive learning with privacy guarantees

Chatalic¹,

Schellekens²,

Houssiau

et al. 2021

Information and Inference: A Journal of the IMA

View full text Add to dashboard Cite

This work addresses the problem of learning from large collections of data with privacy guarantees. The compressive learning framework proposes to deal with the large scale of datasets by compressing them into a single vector of generalized random moments, called a sketch vector, from which the learning task is then performed. We provide sharp bounds on the so-called sensitivity of this sketching mechanism. This allows us to leverage standard techniques to ensure differential privacy—a well-established formalism for defining and quantifying the privacy of a random mechanism—by adding Laplace of Gaussian noise to the sketch. We combine these standard mechanisms with a new feature subsampling mechanism, which reduces the computational cost without damaging privacy. The overall framework is applied to the tasks of Gaussian modeling, k-means clustering and principal component analysis, for which sharp privacy bounds are derived. Empirically, the quality (for subsequent learning) of the compressed representation produced by our mechanism is strongly related with the induced noise level, for which we give analytical expressions.

show abstract

Section: Definition 2 (K-means Clustering Task) Given An Integer > 0 K-means Clustering Consists In Finding Centroidsmentioning

confidence: 99%

Compressive learning with privacy guarantees

Chatalic¹,

Schellekens²,

Houssiau

et al. 2021

Information and Inference: A Journal of the IMA

View full text Add to dashboard Cite

show abstract

“…The parameters can then be extracted by optimizing a cost function as explained later. This approach has been applied to audio sourceseparation [12] as well as speaker verification [10] (see Box 4), for which it was shown that 1000 hours of speech can be compressed down to a few kilobytes without loss of verification performance.…”

Section: C) Gaussian-mixture Modelingmentioning

confidence: 99%

“…In [10] and [12], applications of sketched learning are demonstrated on speaker verification (see Box 4) and source separation.…”

Section: Learning From a Sketchmentioning

confidence: 99%

“…Note that (13) corresponds precisely to the sketch (1) with the random Fourier (RF) feature map Φ(•) from (3). Taking a statistical perspective, the components z j in (12) can also be recognized as samples of the characteristic function of p X . Recall that, for a density p, the characteristic function Ψ p is defined as (the complex conjugate of) its Fourier transform, i.e.,…”

Section: Randomized Generalized Momentsmentioning

confidence: 99%

See 1 more Smart Citation

Sketching Datasets for Large-Scale Learning (long version)

Gribonval,

Chatalic,

Keriven

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

This article considers "sketched learning," or "compressive learning," an approach to large-scale machine learning where datasets are massively compressed before learning (e.g., clustering, classification, or regression) is performed. In particular, a "sketch" is first constructed by computing carefully chosen nonlinear random features (e.g., random Fourier features) and averaging them over the whole dataset. Parameters are then learned from the sketch, without access to the original dataset. This article surveys the current state-ofthe-art in sketched learning, including the main concepts and algorithms, their connections with established signal-processing methods, existing theoretical guarantees-on both information preservation and privacy preservation, and important open problems.Big data can be a blessing: with very large training datasets it becomes possible to perform complex learning tasks with unprecedented accuracy. Yet, this improved performance comes at the price of enormous computational challenges. Thus, one may wonder: Is it possible to leverage the information content of huge datasets while keeping computational resources under control? Can this also help solve some of the privacy issues raised by large-scale learning? This is the ambition of sketched learning-or compressive learning-where the data is massively compressed before learning. Here, a "sketch" is first constructed by computing carefully chosen nonlinear random features (e.g., random Fourier features) and averaging them over the whole dataset. Parameters are then learned from the sketch, without access to the original dataset. This article surveys the current state-of-the-art in sketched learning, including the main concepts and algorithms; their connections with established signal-processing methods; existing theoretical guarantees, on both information preservation and privacy preservation; and important open problems.

show abstract

“…An algorithm for learning patch prior from a sketch : LR-COMP (Low-Rank Continuous Orthogonal Matching Pursuit). Problem (4.7) can be solved approximately using the greedy Compressive Learning OMP called CL-OMP and a variation of CL-OMP called CL-OMP with Replacement (CL-OMPR) [25,26]. These algorithms are based on the Matching Pursuit [34], Orthogonal Matching Pursuit [39] and Orthogonal Matching Pursuit with Replacement [23] for classical compressive sensing, which handle sparse approximation problems.…”

mentioning

confidence: 99%

Compressive Learning for Patch-Based Image Denoising

Shi¹,

Traonmilin²,

Aujol³

2022

SIAM J. Imaging Sci.

View full text Add to dashboard Cite

The Expected Patch Log-Likelihood algorithm (EPLL) and its extensions have shown good performances for image denoising. The prior model used by EPLL is usually a Gaussian Mixture Model (GMM) estimated from a database of image patches. Classical mixture model estimation methods face computational issues as the high dimensionality of the problem requires training on large datasets. In this work, we adapt a compressive statistical learning framework to carry out the GMM estimation. With this method, called sketching, we estimate models from a compressive representation (the sketch) of the training patches. The cost of estimating the prior from the sketch no longer depends on the number of items in the original large database. To accelerate further the estimation, we add another dimension reduction technique (low-rank modeling of the covariance matrices) to the compressing learning framework. To demonstrate the advantages of our method, we test it on real large-scale data. We show that we can produce denoising performances similar to performances obtained with models estimated from the original training database using GMM priors learned from the sketch with improved execution times.

show abstract

Blind Source Separation Using Mixtures of Alpha-Stable Distributions

Cited by 5 publications

References 28 publications

Compressive learning with privacy guarantees

Compressive learning with privacy guarantees

Sketching Datasets for Large-Scale Learning (long version)

Compressive Learning for Patch-Based Image Denoising

Contact Info

Product

Resources

About