Optimal Permutation Recovery in Permuted Monotone Matrix Model

Abstract

Motivated by applications in metagenomics, we consider the permuted monotone matrix model Y = ΘΠ + Z, where Y ∈ R n×p is observed, Θ ∈ R n×p is an unknown signal matrix with monotone rows, Π ∈ R p×p is an unknown permutation matrix, and Z ∈ R n×p is the noise matrix. This paper studies the estimation of the extreme values associated to the signal matrix Θ, including its first and last columns, as well as their difference (the range vector). Treating these estimation problems as compound decision problems, minimax rate-optimal and adaptive estimators are constructed using spectral column sorting. Novel techniques that can be effective in estimating an arbitrary high-dimensional nonlinear operator are developed to establish minimax lower bounds, including generalized Le Cam's method and Fano's method. Numerical experiments using simulated and synthetic microbiome metagenomic data are presented, showing the superiority of the proposed methods over the alternatives. The methods are illustrated by comparing the growth rates of gut bacteria between inflammatory bowel disease patients and normal controls.

Permuted and Unlinked Monotone Regression in $\mathbb{R}^d$: an approach based on mixture modeling and optimal transport

“….) appear rather straightforward, and are not pursued in this paper to simplify the exposition and to facilitate the comparison to related results in previous literature, specifically [15,25,26].…”

“…, x d ) = d j=1 ψ f * j (x j ). Permutation recovery in the presence of noise based on the solution of a linear assignment problem associated with X n and Y n is shown to succeed if a certain minimum signal condition similar to conditions in related papers [15,25,26,27] is met. As a byproduct, the result on permutation recovery herein yields the novel insight that the unlabeled sensing problem in [15] can be solved efficiently whenever the unknown linear transformation is positive (semi)-definite.…”

“…Connection to recovery results in the "permuted monotone matrix model". The paper [26] considers the model…”

“…As elaborated in more detail in §3.1 below, the setup (1) also arises in matrix estimation problems, in which a noisy rowpermuted version of a matrix, whose columns exhibit the same ordering pattern (decreasing or increasing), is observed. The papers [25,26,27] discuss applications in statistical seriation [28] and microbiome data analysis. Finally, model (1) bears a relation to linkage attacks in the literature on data privacy [29,30]: here, Y n may represent (anonymized) sensitive data while an adversary holds auxiliary data X n along with identifiers (e.g., individuals' names) and tries to leverage the functional relationship between the two data sets to guess the values of the sensitive attributes contained in Y n for each or a subset of the identifiers.…”

“…. , f * d ) with f * j increasing on R, 1 ≤ j ≤ d, as studied in [25,26,27] are included, corresponding to component-wise separable additive (strictly) convex functions of the form ψ f * (x 1 , . .…”