Graph Tikhonov Regularization and Interpolation Via Random Spanning Forests

Abstract

Novel Monte Carlo estimators are proposed to solve both the Tikhonov regularization (TR) and the interpolation problems on graphs. These estimators are based on random spanning forests (RSF), the theoretical properties of which enable to analyze the estimators' theoretical mean and variance. We also show how to perform hyperparameter tuning for these RSFbased estimators. TR is a component in many well-known algorithms, and we show how the proposed estimators can be easily adapted to avoid expensive intermediate steps in generalized semi-supervised learning, label propagation, Newton's method and iteratively reweighted least squares. In the experiments, we illustrate the proposed methods on several problems and provide observations on their run time.

Loop-erased partitioning of a graph: mean-field analysis

“…Recently, a number of algorithms based on these results have been proposed as tools for tackling different problems in data science. These applications include: wavelets basis and filters for signal processing on graphs [3,29,28], trace estimation [10], network renormalization [4,5], centrality measures [16] and statistical learning [9]. The fact that random rooted forests proved a powerful tool in such different applied areas, was one of the staring motivation for the analysis of the partition measure in (1.9).…”

Variance Reduction in Stochastic Methods for Large-Scale Regularized Least-Squares Problems

“…The result is another unbiased estimator, one that is guaranteed to have lower variance. We apply our technique to Tikhonov regularization on graphs, where DPPbased estimators are particularly well-adapted [5,6]. As in standard applications of gradient descent, most of the difficulty consists in determining how large the gradient descent step should be.…”

“…A "graph signal" is a vector of measurements associated with the nodes of a graph, for instance brain activity in n brain regions, where the graph models neural connectivity across regions. GTR also occurs as a subproblem in other methods, like semi-supervised learning [6], which is why finding an efficient approximation algorithm is of high interest.…”

“…As an alternative stochastic method for GTR, we have proposed in previous works unbiased Monte Carlo estimators for approximating x which also scale linearly with m [5,6]. Our method can be viewed as a DPP-based subsampling of the rows of a matrix, but the case of GTR is particularly favourable.…”

“…The appropriate DPP is the random forest process [10], which is easy to sample from, and its analytic properties enable variance reduction via conditional Monte Carlo. The resulting estimator, called x in [5,6], is unbiased (E(x) = x), can be obtained in O(m) elementary operations, and takes a very simple form: based on the partition formed by a random spanning forest, it averages the signal in each part of the partition. We repeat this process N times and average to get a denoised signal.…”