(Nearly) Sample-Optimal Sparse Fourier Transform in Any Dimension; RIPless and Filterless

Nakos, Vasileios; Song, Zhao; Wang, Zhengyu

doi:10.1109/focs.2019.00092

Cited by 14 publications

(5 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, while those filters are particularly efficient in low dimensions, their performance deteriorates when the number of dimensions increases: indeed, a d-dimensional ∞ box has 2 d faces, and hence this approach suffers inevitably from the curse of dimensionality. On the other hand, an unstructured collection of O(k • poly(log N )) samples [CT06,NSW19] suffice, showing that the sample complexity is dimension-independent; the cost that one needs to pay, however, is Ω(N ) running time.…”

Section: Previous Techniquesmentioning

confidence: 99%

Traversing the FFT Computation Tree for Dimension-Independent Sparse Fourier Transforms

Bringmann¹,

Kapralov²,

Макаров³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Computing the dominant Fourier coefficients of a function/vector is a common task in many fields, such as signal processing, learning theory and computational complexity. In the Sparse Fast Fourier Transform (Sparse FFT) problem, one is given oracle access to a d-dimensional vector x of size N , and is asked to compute the best k-term approximation of x, the Discrete Fourier Transform of x, quickly and using few samples of the input vector x. Sparse FFT has received a significant amount of attention over the past years. However, while the sample complexity of the problem is quite well understood, all previous approaches either suffer from an exponential dependence of runtime on the dimension d or can only tolerate a trivial amount of noise. This is in sharp contrast with the classical FFT algorithm of Cooley and Tukey, which is stable and completely insensitive to the dimension of the input vector: its runtime is O(N log N ) in any dimension d.In this work, we make progress in high-dimensional FFTs by introducing a new Sparse FT toolkit and using it to obtain new algorithms, both on the exact, as well as in the case of bounded 2 noise. This toolkit includes i) a new strategy for exploring a pruned FFT computation tree that reduces the cost of filtering, ii) new structural properties of adaptive aliasing filters recently introduced by Kapralov, Velingker and Zandieh'SODA'19, and iii) a novel lazy estimation argument, suited to reducing the cost of estimation in FFT tree-traversal approaches. Our robust algorithm can be viewed as a highly optimized sparse, stable extension of the Cooley-Tukey FFT algorithm.Finally, we explain the barriers we have faced by proving a conditional quadratic lower bound on the running time of the well-studied non-equispaced Fourier transform problem. Among other consequences, this resolves a natural and frequently asked question in computational Fourier transforms, see for example Problem 21 from IITK Workshop on Algorithms for Data Streams, Kanpur 2006. Lastly, we provide a preliminary experimental evaluation comparing the runtime of our algorithm to FFTW and SFFT 2.0.

show abstract

Section: Previous Techniquesmentioning

confidence: 99%

Traversing the FFT Computation Tree for Dimension-Independent Sparse Fourier Transforms

Bringmann¹,

Kapralov²,

Макаров³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…One of our main algorithmic tools will be the univariate (continuous) Fourier transform, as a way to estimate Fourier moments of our distribution. In recent years, the question of learning the Fourier transform of a function has attracted a considerable amount of interest in theoretical computer science [9,20,22,24,25,33,35,38]. Our application is somewhat different in that we have explicit access to the function we will take the Fourier transform of.…”

Section: List-decodable Regressionmentioning

confidence: 99%

Learning mixtures of linear regressions in subexponential time via Fourier moments

Chen

Song

2020

Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing

Self Cite

View full text Add to dashboard Cite

We consider the problem of learning a mixture of linear regressions (MLRs). An MLR is specified by k nonnegative mixing weights p 1 ,. .. ,p k summing to 1, and k unknown regressors w 1 , ...,w k ∈ R d. A sample from the MLR is drawn by sampling i with probability p i , then outputting (x,y) where y = ⟨x,w i ⟩ +η, where η ∼ N (0,ς 2) for noise rate ς. Mixtures of linear regressions are a popular generative model and have been studied extensively in machine learning and theoretical computer science. However, all previous algorithms for learning the parameters of an MLR require running time and sample complexity scaling exponentially with k. In this paper, we give the first algorithm for learning an MLR that runs in time which is sub-exponential in k. Specifically, we give an algorithm which runs in time O (d) • exp(O (√ k)) and outputs the parameters of the MLR to high accuracy, even in the presence of nontrivial regression noise. We demonstrate a new method that we call Fourier moment descent which uses univariate density estimation and low-degree moments of the Fourier transform of suitable univariate projections of the MLR to iteratively refine our estimate of the parameters. To the best of our knowledge, these techniques have never been used in the context of high dimensional distribution learning, and may be of independent interest. We also show that our techniques can be used to give a sub-exponential time algorithm for a natural hard instance of the subspace clustering problem, which we call learning mixtures of hyperplanes. CCS CONCEPTS • Theory of computation → Unsupervised learning and clustering; • Mathematics of computing → Probability and statistics.

show abstract

“…Due to its theoretical elegance and accurate modeling of real-world streaming computing systems (such as Spark Streaming [ZDL + 12]), the streaming model has attracted a lot of attention in the past decades. Spaceefficient streaming algorithms were developed for a series of fundamental problems including e.g., frequency moments estimation [AMS99, IW05, BO10, JW19], p sampling [MW10, AKO11, JW21], clustering [FL11, GM16, BFL + 17], coverage [BMKK14, CW16, ER16, SG09], diversity maximization [Ind04,CPPU17], sparse recovery [NSW19,NS19], low rank matrix approximation [GP14, Lib13, BWZ16, SWZ17], graph problems [FKM + 05, AGM13, AGK14, SW15, LSZ20, CKP + 21]. We refer readers to a survey [Mut05] for more streaming algorithms and applications.…”

Section: Introductionmentioning

confidence: 99%

Improved Sliding Window Algorithms for Clustering and Coverage via Bucketing-Based Sketches

Epasto¹,

Mahdian²,

Mirrokni³

et al. 2021

Preprint

View full text Add to dashboard Cite

Streaming computation plays an important role in large-scale data analysis. The sliding window model is a model of streaming computation which also captures the recency of the data. In this model, data arrives one item at a time, but only the latest W data items are considered for a particular problem. The goal is to output a good solution at the end of the stream by maintaining a small summary during the stream.In this work, we propose a new algorithmic framework for designing efficient sliding window algorithms via bucketing-based sketches. Based on this new framework, we develop space-efficient sliding window algorithms for k-cover, k-clustering and diversity maximization problems. For each of the above problems, our algorithm achieves (1 ± )-approximation. Compared with the previous work, it improves both the approximation ratio and the space.

show abstract

(Nearly) Sample-Optimal Sparse Fourier Transform in Any Dimension; RIPless and Filterless

Cited by 14 publications

References 45 publications

Traversing the FFT Computation Tree for Dimension-Independent Sparse Fourier Transforms

Traversing the FFT Computation Tree for Dimension-Independent Sparse Fourier Transforms

Learning mixtures of linear regressions in subexponential time via Fourier moments

Improved Sliding Window Algorithms for Clustering and Coverage via Bucketing-Based Sketches

Contact Info

Product

Resources

About