An economic theory of statistical testing

Tetenov, Aleksey

doi:10.1920/wp.cem.2016.5016

Cited by 11 publications

(20 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…36 While in models of optimal persuasion a la Rayo and Segal (2010) and Kamenica and Gentzkow (2011) the sender can freely commit to give any signal to the receiver, our researcher is able to partly affect the signal observed by the receiver only through a hidden sampling/assignment choice. 37 While persuasion benefits the sender in Kamenica and Gentzkow (2011), our researcher faces a commitment problem and thus can be harmed by 32 See also Tetenov (2015) for a recent development. 33 See also Dahm et al (2009) andHenry (2009) for related approaches to disclosure in science.…”

Section: Resultsmentioning

confidence: 99%

Persuasion Bias in Science: Can Economics Help?

Tillio

Ottaviani

Sørensen

2017

The Economic Journal

View full text Add to dashboard Cite

We investigate the impact of conflicts of interests on randomised controlled trials in a game‐theoretic framework. A researcher seeks to persuade an evaluator that the causal effect of a treatment outweighs its cost, to justify acceptance. The researcher can use private information to manipulate the experiment in three alternative ways: (i) sampling subjects based on their treatment effect, (ii) assigning subjects to treatment based on their baseline outcome, or (iii) selectively reporting experimental outcomes. The resulting biases have different welfare implications: for sufficiently high acceptance cost, in our binary illustration the evaluator loses in cases (i) and (iii) but benefits in case (ii).

show abstract

Section: Resultsmentioning

confidence: 99%

Persuasion Bias in Science: Can Economics Help?

Tillio

Ottaviani

Sørensen

2017

The Economic Journal

View full text Add to dashboard Cite

show abstract

“…Kasy (2016) showed how deterministic assignment rules improve inference over randomization conditional on covariates. Tetenov (2016) analyzed an evaluator's optimal commitment to a decision rule when privately informed researchers select into costly testing. Banerjee et al (2020) analyzed experiment design by an ambiguity‐averse researcher facing an adversarial evaluator.…”

Section: Application To Strategic Settingsmentioning

confidence: 99%

Strategic Sample Selection

Tillio

Ottaviani

Sørensen

2021

ECTA

View full text Add to dashboard Cite

Are the highest sample realizations selected from a larger presample more or less informative than the same amount of random data? Developing multivariate accuracy for interval dominance ordered preferences, we show that sample selection always benefits (or always harms) a decision maker if the reverse hazard rate of the data distribution is log‐supermodular (or log‐submodular), as in location experiments with normal noise. We find nonpathological conditions under which the information contained in the winning bids of a symmetric auction decreases in the number of bidders. Exploiting extreme value theory, we quantify the limit amount of information revealed when the presample size (number of bidders) goes to infinity. In a model of equilibrium persuasion with costly information, we derive implications for the optimal design of selected experiments when selection is made by an examinee, a biased researcher, or contending sides with the peremptory challenge right to eliminate a number of jurors.

show abstract

“…In a follow-up paper, Tetenov (2016) continues to focus on the above microeconomic treatment decision framework and derives the optimal significance level of a test as a function of test power and costs of treatment adoption. While this moves closer to our framework of viewing the significance level of a test as a tuning parameter to be chosen by optimising a loss function, it is very specific to the treatment problem at hand.…”

Section: Recent Work Bymentioning

confidence: 99%

A new approach for detecting shifts in forecast accuracy

Chiu

Hayes

Kapetanios

et al. 2019

International Journal of Forecasting

View full text Add to dashboard Cite

Forecasts play a critical role at inflation targeting central banks, such as the Bank of England. Breaks in the forecast performance of a model can potentially incur important policy costs. Commonly used statistical procedures, however, implicitly put a lot of weight on type I errors (or false positives), which result in a relatively low power of tests to identify forecast breakdowns in small samples. We develop a procedure which aims at capturing the policy cost of missing a break. We use data-based rules to find the test size that optimally trades off the costs associated with false positives with those that can result from a break going undetected for too long. In so doing, we also explicitly study forecast errors as a multivariate system. The covariance between forecast errors for different series, though often overlooked in the forecasting literature, not only enables us to consider testing in a multivariate setting but also increases the test power. As a result, we can tailor the choice of the critical values for each series not only to the in-sample properties of each series but also to how the series for forecast errors covary.

show abstract

An economic theory of statistical testing

Cited by 11 publications

References 24 publications

Persuasion Bias in Science: Can Economics Help?

Persuasion Bias in Science: Can Economics Help?

Strategic Sample Selection

A new approach for detecting shifts in forecast accuracy

Contact Info

Product

Resources

About