Calibration: Respice, Adspice, Prospice

Foster, Dean P.; Vohra, Rakesh

doi:10.1017/cbo9781139060011.014

Cited by 7 publications

(5 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A growing literature studies whether strategic experts can avoid rejection (see, among several contributions, Al-Najjar and Weinstein (2008), Al-Najjar, Smorodinsky, Sandroni and Weinstein (2010), Babaio , Blumrosen, Lambert and Reingold (2011), Cesa-Bianchi and Lugosi (2006), Chassang (2013), Dekel and Feinberg (2006), Feinberg and Stewart (2008), Feinberg and Lambert (2014), Fortnow and Vohra (2009), Foster and Vohra (1998), Fudenberg and Levine (1999), Gradwohl and Salant (2011), Gradwohl and Shmaya (2013), Hu and Shmaya (2013), Lehrer (2001), Olszewski and Peski (2011), Olszewski andSandroni (2008,2009a-b), Sandroni (2003), Shmaya (2008), Stewart (2011), and Vovk and Shafer (2005)). For a review, see Foster and Vohra (2013) and Olszewski (2011).…”

Section: A Related Literaturementioning

confidence: 99%

Claim Validation

Al-Najjar

Pomatto

Sandroni

2014

American Economic Review

View full text Add to dashboard Cite

Hume (1748) challenged the idea that a general claim (e.g., “all swans are white”) can be validated by empirical evidence, no matter how compelling. We examine this issue from the perspective of a tester who must accept or reject the forecasts of a potential expert. If experts can be skeptical about the validity of claims then they can evade rejection strategically. In contrast, if experts are required to conclude that claims backed by sufficient evidence are likely to be true, then they can be tested and rejected. These results provide an economic rationale for claim validation based on incentive problems. (JEL D82)

show abstract

Section: A Related Literaturementioning

confidence: 99%

Claim Validation

Al-Najjar

Pomatto

Sandroni

2014

American Economic Review

View full text Add to dashboard Cite

show abstract

“…Foster and Vohra (2013) and Olszewski (2015) survey the literature on testing strategic forecasters 3 . In this section, we comment on those papers that are closer to the present work.…”

Section: Introductionmentioning

confidence: 99%

Testable forecasts

Pomatto¹

2021

View full text Add to dashboard Cite

Predictions about the future are commonly evaluated through statistical tests. As shown by recent literature, many known tests are subject to adverse selection problems and cannot discriminate between forecasters who are competent and forecasters who are uninformed but predict strategically. We consider a framework where forecasters' predictions must be consistent with a paradigm, a set of candidate probability laws for the stochastic process of interest. This paper presents necessary and sufficient conditions on the paradigm under which it is possible to discriminate between informed and uninformed forecasters. We show that optimal tests take the form of likelihood‐ratio tests comparing forecasters' predictions against the predictions of a hypothetical Bayesian outside observer. In addition, the paper illustrates a new connection between the problem of testing strategic forecasters and the classical Neyman–Pearson paradigm of hypothesis testing.

show abstract

“…The literature on testing forecasters (for surveys, see Foster and Vohra 2011 , Olszewski 2015 ) shares with our model the aim of designing a test to uncover the type of a strategic agent (an “expert”). In that literature, the expert makes probabilistic forecasts about an unknown stochastic process, and the principal seeks to determine whether the expert knows the true probabilities or is completely ignorant.…”

Section: Introductionmentioning

confidence: 99%

Optimal adaptive testing: Informativeness and incentives

Deb¹,

Stewart²

2018

Theoretical Economics

View full text Add to dashboard Cite

We introduce a learning framework in which a principal seeks to determine the ability of a strategic agent. The principal assigns a test consisting of a finite sequence of tasks. The test is adaptive: each task that is assigned can depend on the agent's past performance. The probability of success on a task is jointly determined by the agent's privately known ability and an unobserved effort level that he chooses to maximize the probability of passing the test. We identify a simple monotonicity condition under which the principal always employs the most (statistically) informative task in the optimal adaptive test. Conversely, whenever the condition is violated, we show that there are cases in which the principal strictly prefers to use less informative tasks. several anonymous referees, and various seminar and conference participants for helpful comments and suggestions. Rya Sciban and Young Wu provided excellent research assistance. We are very grateful for the financial support provided by the Social Sciences and Humanities Research Council of Canada.1 Learning a worker's ability by observing their performance on differentially informative tasks is a problem that goes back to at least Prescott and Visscher (1980). 2 This assumption is consistent with the observation in Baker et al. (1988) that in many organizations, promotion is the only means used for providing incentives. 3 This feature of the optimal test is reminiscent of the optimal contract in the (two outcome version of the) dynamic, pure moral hazard framework of Hölmstrom and Milgrom (1987), where the agent's overall compensation depends only on the number-and not the order-of successes. 4 Prasad (2009) and Ferreira and Sah (2012) are recent examples of models where workers can be either generalists or specialists.

show abstract

Calibration: Respice, Adspice, Prospice

Cited by 7 publications

References 39 publications

Claim Validation

Claim Validation

Testable forecasts

Optimal adaptive testing: Informativeness and incentives

Contact Info

Product

Resources

About