Multiple testing under dependence via graphical models

Liu, Jie; Zhang, Chunming; Page, David L.

doi:10.1214/16-aoas956

Cited by 18 publications

(20 citation statements)

References 65 publications

(62 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the typical MLE procedures require that we first specify the log likelihood function, and then find the MLE of the parameters, it does not mean that we cannot find the MLE for MRFs. Indeed, a few recent algorithms [7,37,9,31,32, 1] make use of the concavity of the MRF's log likelihood function and find the MLE via gradient ascent with both satisfactory empirical performance [12,11,10,13,14,15] and convergence properties [35,36,3,27,30]. With the availability of MLE of the parameters, we further demonstrate that we are able to recover the intractable likelihood function to some precision from the MLE of the parameters.…”

Section: Discussionmentioning

confidence: 99%

“…In the standard single-variable exchange algorithm, s samples need to be generated from P (X; θ * ) in each MH step when we propose θ * , and the MH ratio is calculated as (14). The motivation of the persistent Markov chains algorithm [5] is that although the proposed θ * is different from θ, θ * is usually not far away from θ because MH algorithms usually require proposing small changes so as to maintain a high acceptance rate.…”

Section: Exchange Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

MLE-induced Likelihood for Markov Random Fields

Liu,

Zheng

2018

Preprint

Self Cite

View full text Add to dashboard Cite

Due to the intractable partition function, the exact likelihood function for a Markov random field (MRF), in many situations, can only be approximated. Major approximation approaches include pseudolikelihood [2] and Laplace approximation [33]. In this paper, we propose a novel way of approximating the likelihood function through first approximating the marginal likelihood functions of individual parameters and then reconstructing the joint likelihood function from these marginal likelihood functions. For approximating the marginal likelihood functions, we derive a particular likelihood function from a modified scenario of coin tossing which is useful for capturing how one parameter interacts with the remaining parameters in the likelihood function. For reconstructing the joint likelihood function, we use an appropriate copula to link up these marginal likelihood functions. Numerical investigation suggests the superior performance of our approach. Especially as the size of the MRF increases, both the numerical performance and the computational cost of our approach remain consistently satisfactory, whereas Laplace approximation deteriorates and pseudolikelihood becomes computationally unbearable.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Exchange Algorithmmentioning

confidence: 99%

MLE-induced Likelihood for Markov Random Fields

Liu,

Zheng

2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…To tackle statistical dependence in multiple testing, some studies have taken a probabilistic approach by modeling the data as stochastic processes with nontrivial dependence structure. They use specific models such as hidden Markov models [55,56], Markov random fields [35,38], or Gaussian random fields [9,48]. The main strength of these models is their tractability.…”

Section: Hypothesis Accept Reject Totalmentioning

confidence: 99%

False Discovery Variance Reduction in Large Scale Simultaneous Hypothesis Tests

Rayaprolu

Chi

2020

Methodol Comput Appl Probab

View full text Add to dashboard Cite

Statistical dependence between hypotheses poses a significant challenge to the stability of large scale multiple hypotheses testing. Ignoring it often results in an unacceptably large spread in the false positive proportion even though the average value is acceptable [21,39,40,49]. However, the statistical dependence structure of data is often unknown. Using a generic signalprocessing model, Bayesian multiple testing, and simulations, we demonstrate that the variance of the false positive proportion can be substantially reduced even under unknown short range dependence. We do this by modeling the data generating process as a stationary ergodic binary signal process embedded in noisy observations. We derive conditional probabilities needed for the Bayesian multiple testing by incorporating nearby observations into a second order Taylor series approximation. Simulations under general conditions are carried out to assess the validity and the variance reduction of the approach. Along the way, we address the problem of sampling a random Markov matrix with specified stationary distribution and lower bounds on the top absolute eigenvalues, which is of interest in its own right.

show abstract

“…While FDR control under dependence is known to be a challenging issue, as the most classical results use independence or positive dependence of the test statistics Benjamini and Hochberg (1995); Benjamini and Yekutieli (2001), these models circumvent this difficulty by assuming that the test statistics are independent conditionally on the structure. Former studies include group structure Sun and Cai (2009) and Markov structures Cai and Sun (2009); Liu et al (2016). These methods have the strong advantage to both control the FDR under dependencies while allowing more detections than procedures ignoring the structure, as the BH procedure do.…”

Section: False Discovery Ratementioning

confidence: 99%

Graph inference with clustering and false discovery rate control

Rebafka,

Roquain,

Villers

2019

Preprint

View full text Add to dashboard Cite

In this paper, a noisy version of the stochastic block model (NSBM) is introduced and we investigate the three following statistical inferences in this model: estimation of the model parameters, clustering of the nodes and identification of the underlying graph. While the two first inferences are done by using a variational expectation-maximization (VEM) algorithm, the graph inference is done by controlling the false discovery rate (FDR), that is, the average proportion of errors among the edges declared significant, and by maximizing the true discovery rate (TDR), that is, the average proportion of edges declared significant among the true edges. Provided that the VEM algorithm provides reliable parameter estimates and clustering, we theoretically show that our procedure does control the FDR while satisfying an optimal TDR property, up to remainder terms that become small when the size of the graph grows. Numerical experiments show that our method outperforms the classical FDR controlling methods that ignore the underlying SBM topology. In addition, these simulations demonstrate that the FDR/TDR properties of our method are robust to model mis-specification, that is, are essentially maintained outside our model.

show abstract

Multiple testing under dependence via graphical models

Cited by 18 publications

References 65 publications

MLE-induced Likelihood for Markov Random Fields

MLE-induced Likelihood for Markov Random Fields

False Discovery Variance Reduction in Large Scale Simultaneous Hypothesis Tests

Graph inference with clustering and false discovery rate control

Contact Info

Product

Resources

About