Testing in the Presence of Nuisance Parameters: Some Comments on Tests Post-Model-Selection and Random Critical Values

Leeb, Hannes; Pötscher, Benedikt M.

doi:10.1007/978-3-319-41573-4_4

Cited by 22 publications

(10 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many authors have studied hypothesis testing problems in which a nuisance parameter is only identifiable under the alternative (e.g., Davies, 1977Davies, , 1987Davies, , 2002Hansen, 1996). Here we encounter the situation where nuisance parameters appear only in the null, so calibration of the test statistic may potentially depend on L. Leeb and Pötscher (2017) have studied a post-selection calibration method that uses estimates of such nuisance parameters, but, as we will see, our approach leads to an asymptotically pivotal estimator of τ max without the need to estimate L.…”

Section: Given Independent Observationsmentioning

confidence: 99%

Significance testing for canonical correlation analysis in high dimensions

McKeague

Zhang

2020

Preprint

View full text Add to dashboard Cite

We consider the problem of testing for the presence of linear relationships between large sets of random variables based on a post-selection inference approach to canonical correlation analysis. The challenge is to adjust for the selection of subsets of variables having linear combinations with maximal sample correlation. To this end, we construct a stabilized onestep estimator of the euclidean-norm of the canonical correlations maximized over subsets of variables of pre-specified cardinality. This estimator is shown to be consistent for its target parameter and asymptotically normal provided the dimensions of the variables do not grow too quickly with sample size. We also develop a greedy search algorithm to accurately compute the estimator, leading to a computationally tractable omnibus test for the global null hypothesis that there are no linear relationships between any subsets of variables having the pre-specified cardinality. Further, we develop a confidence interval for the target parameter that takes the variable selection into account.

show abstract

Section: Given Independent Observationsmentioning

confidence: 99%

Significance testing for canonical correlation analysis in high dimensions

McKeague

Zhang

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…A natural approach to control size in the presence of model selection is to take a least favorable (LF) approach and to use the largest critical value across all values for the nuisance parameter (e.g., D. W. K. Andrews & Guggenberger, 2009;Leeb & Pötscher, 2017). However, it is well-known that this worst-case approach can exhibit poor power properties.…”

Section: A51 Model Selection With a Bonferroni-style Correctionmentioning

confidence: 99%

Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments

Muralidharan

Romero²,

Wüthrich

2019

View full text Add to dashboard Cite

Factorial designs are widely used for studying multiple treatments in one experiment. While "long" model t-tests provide valid inferences, "short" model t-tests (ignoring interactions) yield higher power if interactions are zero, but incorrect inferences otherwise. Of 27 factorial experiments published in top-5 journals (2007-2017), 19 use the short model. After including all interactions, over half their results lose significance. Modest local power improvements over the long model are possible, but with lower power for most values of the interaction. If interactions are not of interest, leaving the interaction cells empty yields valid inferences and global power improvements.

show abstract

“…There is a large literature on inference after model selection, including Pötscher (1991), Kabaila (1995Kabaila ( , 1998, Pötscher and Leeb (2009), and Leeb and Pötscher (2003, 2008, 2017. These articles point out that the coverage probabilities of naive confidence intervals are lower than the nominal values.…”

Section: Introductionmentioning

confidence: 99%

Inference After Model Averaging in Linear Regression Models

Zhang

Liu

2018

Econom. Theory

View full text Add to dashboard Cite

This article considers the problem of inference for nested least squares averaging estimators. We study the asymptotic behavior of the Mallows model averaging estimator (MMA; Hansen, 2007) and the jackknife model averaging estimator (JMA; Hansen and Racine, 2012) under the standard asymptotics with fixed parameters setup. We find that both MMA and JMA estimators asymptotically assign zero weight to the under-fitted models, and MMA and JMA weights of just-fitted and over-fitted models are asymptotically random. Building on the asymptotic behavior of model weights, we derive the asymptotic distributions of MMA and JMA estimators and propose a simulation-based confidence interval for the least squares averaging estimator. Monte Carlo simulations show that the coverage probabilities of proposed confidence intervals achieve the nominal level.

show abstract

Testing in the Presence of Nuisance Parameters: Some Comments on Tests Post-Model-Selection and Random Critical Values

Cited by 22 publications

References 13 publications

Significance testing for canonical correlation analysis in high dimensions

Significance testing for canonical correlation analysis in high dimensions

Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments

Inference After Model Averaging in Linear Regression Models

Contact Info

Product

Resources

About