Aaditya Ramdas’s Contribution to the Discussion of ‘Testing by Betting: A Strategy for Statistical and Scientific Communication’ by Glenn Shafer

Abstract

The most widely used concept of statistical inference-the p -value-is too complicated for effective communication to a wide audience (Gigerenzer, 2018 ; McShane & Gal, 2017 ). This paper introduces a simpler way of reporting statistical evidence: report the outcome of a bet against the null hypothesis. This leads to a new role for likelihood, to alternatives to power and confidence, and to a framework for meta-analysis that accommodates both planned and opportunistic testing of statistical hypotheses and probabilistic forecasts.Testing a hypothesized probability distribution by betting is straightforward. We select a nonnegative payoff and buy it for its hypothesized expected value. If this bet multiplies the money it risks by a large factor, we have evidence against the hypothesis, and the factor measures the strength of this evidence. Multiplying our money by 5 might merit attention; multiplying it by 100 or by 1000 might be considered conclusive.

Anytime-Valid Confidence Sequences in an Enterprise A/B Testing Platform

“…Subsequent work focused on extending CSs to richer nonparametric problems, [11] such as those for bounded random variables [15,20,31]. See Ramdas et al [22] for a more detailed survey. Most of the CS literature focuses on non-asymptotic methods which have three major disadvantages even for fixed-horizon settings: (a) they require strong assumptions, such as a parametric model or known moment generating functions [11,23,27,30,31], (b) they are typically wider than asymptotic methods based on the central limit theorem, and (c) they take different forms for different problems, whereas the central limit theorem yields a universal and closedform (trivial-to-compute) expression.…”

A composite generalization of Ville’s martingale theorem using e-processes

“…We revisit this notion very soon since it plays a fundamental role in the current paper. The aforementioned discoveries underpin a "game-theoretic" approach to statistics, which in particular yields hypothesis tests and confidence sets that can be continuously monitored and adaptively stopped; see the recent survey [16] and references therein for an elaboration on such "safe anytime-valid inference. "…”