False Discovery and its Control in Low Rank Estimation

Abstract

Summary Models specified by low rank matrices are ubiquitous in contemporary applications. In many of these problem domains, the row–column space structure of a low rank matrix carries information about some underlying phenomenon, and it is of interest in inferential settings to evaluate the extent to which the row–column spaces of an estimated low rank matrix signify discoveries about the phenomenon. However, in contrast with variable selection, we lack a formal framework to assess true or false discoveries in low rank estimation; in particular, the key source of difficulty is that the standard notion of a discovery is a discrete notion that is ill suited to the smooth structure underlying low rank matrices. We address this challenge via a geometric reformulation of the concept of a discovery, which then enables a natural definition in the low rank case. We describe and analyse a generalization of the stability selection method of Meinshausen and Bühlmann to control for false discoveries in low rank estimation, and we demonstrate its utility compared with previous approaches via numerical experiments.

Perturbations and Causality in Gaussian Latent Variable Models

“…Using the relation (30) and a similar analysis as with the proof under Assumptions 1-5 in (10), one can arrive at the conclusion of Lemma 2 with Assumptions 1-5 in (11).…”

“…Further, we examine the effect of the estimated hidden variable on the proteins. Specifically, following [30], we compute the top singular vector of the average projection matrix (computed across the seven settings) onto the 1-dimensional hidden variable subspace. The magnitude of the entries of this singular vector are below 0.015 for all proteins except PKC, P38, and JNK, whose magnitudes are all above 0.5.…”

Learning Exponential Family Graphical Models with Latent Variables using Regularized Conditional Likelihood

“…The methods presented in [14,17] may be described in terms of this matrix, and they were applicable to discrete model selection problems such as graph estimation and variable selection. These ideas were extended recently to low-rank estimation problems based on a geometric reformulation of model selection [21], with a key ingredient being a suitable generalization of the aggregate matrix P graph λ,γ . Specifically, for each L(l) λ,γ let P (l) λ,γ ⊂ S d denote the projection operator onto the column-space of L(l) λ,γ .…”

“…Stage 2: Identifying Model Structure Solving (2.2) or (2.11) with the regularization parameters obtained from the preceding step tends to lead to models that have small type-II error (formally [14] shows that type-II error in graph structure estimation is small under minimal assumptions). However, to also reduce type-I error it is useful to further restrict the models selected based on a more refined form of stability, as described in [17,21]. Specifically, while the approach of [14] considers aggregate variability, the methods in [17,21] suggest selecting a graphical model structure and a latent subspace that are common to a large proportion of the subsamples.…”

“…However, to also reduce type-I error it is useful to further restrict the models selected based on a more refined form of stability, as described in [17,21]. Specifically, while the approach of [14] considers aggregate variability, the methods in [17,21] suggest selecting a graphical model structure and a latent subspace that are common to a large proportion of the subsamples. Concretely, let P graph and P latent represent the average projections over subsamples for the values of the regularization parameters chosen from the previous step (we have suppressed the dependence on λ, γ for notational clarity).…”