Isotonic regression with unknown permutations: Statistics, computation, and adaptation

Abstract

Motivated by models for multiway comparison data, we consider the problem of estimating a coordinate-wise isotonic function on the domain [0, 1] d from noisy observations collected on a uniform lattice, but where the design points have been permuted along each dimension. While the univariate and bivariate versions of this problem have received significant attention, our focus is on the multivariate case d ≥ 3. We study both the minimax risk of estimation (in empirical L 2 loss) and the fundamental limits of adaptation (quantified by the adaptivity index) to a family of piecewise constant functions. We provide a computationally efficient Mirsky partition estimator that is minimax optimal while also achieving the smallest adaptivity index possible for polynomial time procedures. Thus, from a worst-case perspective and in sharp contrast to the bivariate case, the latent permutations in the model do not introduce significant computational difficulties over and above vanilla isotonic regression. On the other hand, the fundamental limits of adaptation are significantly different with and without unknown permutations: Assuming a hardness conjecture from average-case complexity theory, a statistical-computational gap manifests in the former case. In a complementary direction, we show that natural modifications of existing estimators fail to satisfy at least one of the desiderata of optimal worst-case statistical performance, computational efficiency, and fast adaptation. Along the way to showing our results, we improve adaptation results in the special case d = 2 and establish some properties of estimators for vanilla isotonic regression, both of which may be of independent interest.

Optimal detection of the feature matching map in presence of noise and outliers

“…Finally, permutation estimation and related problems have been recently investigated in different contexts such as statistical seriation [15], noisy sorting [33], regression with shuffled data [37,41], isotonic regression and matrices [32,36,30], crowd labeling [40], and recovery of general discrete structure [16].…”

Exact Clustering in Tensor Block Model: Statistical Optimality and Computational Limit

“…Several open questions on HPC detection, in particular, whether HPC detection is equivalently hard as PC detection is discussed in Luo and Zhang (2020a). The HPC detection conjecture has been used to establish the computational limits for a number of problems, including subtensor detection and recovery (Luo and Zhang, 2020b), tensor PCA (Brennan and Bresler, 2020) and isotonic regression for multiway comparison data (Pananjady and Samworth, 2020).…”

Matrix Reordering for Noisy Disordered Matrices: Optimality and Computationally Efficient Algorithms

“…Many statistical seriation problems [6,33,44], that in one way or another aim to find an element in the discrete permutation set optimizing certain objective function, have been studied from various aspects under different settings. These include the well-known consecutive one's problem [29,41,42] that dates back to the 1960s; the feature matching problem [23,38,30] and the noisy ranking problem [11,39,54,20,63,51,55,22]; the matrix seriation problem for various shape-constrained matrices including the monotone or bi-monotone matrices [26,50,47,57], the Robinson matrices [2,27,61,1], and the Monge matrices [36]; and more recently, the seriation problem under the latent space models [31,37]. Many of the existing works have focused on recovering the underlying permutations, estimation of the (disordered) signal structures, or both.…”

“…The tradeoff between computational efficiency and statistical accuracy has been observed in other permutation-related statistical problems such as sparse/submatrix detection [49,17], structured PCA [14,65], permuted isotonic regression [50,57], tensor spectral clustering [46], among many others. In particular, assuming the computational hardness of the wellknown planted clique problem, many of these problems [49,65,17,57,46] have been shown to preserve a regime with fundamental computational barrier; that is, any randomized polynomial-time algorithm must be statistically suboptimal.…”