Uncertainty in the Hot Hand Fallacy: Detecting Streaky Alternatives to Random Bernoulli Sequences

Abstract

We study a class of permutation tests of the randomness of a collection of Bernoulli sequences and their application to analyses of the human tendency to perceive streaks of consecutive successes as overly representative of positive dependence—the hot hand fallacy. In particular, we study permutation tests of the null hypothesis of randomness (i.e., that trials are i.i.d.) based on test statistics that compare the proportion of successes that directly follow k consecutive successes with either the overall proportion of successes or the proportion of successes that directly follow k consecutive failures. We characterize the asymptotic distributions of these test statistics and their permutation distributions under randomness, under a set of general stationary processes, and under a class of Markov chain alternatives, which allow us to derive their local asymptotic power. The results are applied to evaluate the empirical support for the hot hand fallacy provided by four controlled basketball shooting experiments. We establish that substantially larger data sets are required to derive an informative measurement of the deviation from randomness in basketball shooting. In one experiment, for which we were able to obtain data, multiple testing procedures reveal that one shooter exhibits a shooting pattern significantly inconsistent with randomness – supplying strong evidence that basketball shooting is not random for all shooters all of the time. However, we find that the evidence against randomness in this experiment is limited to this shooter. Our results provide a mathematical and statistical foundation for the design and validation of experiments that directly compare deviations from randomness with human beliefs about deviations from randomness, and thereby constitute a direct test of the hot hand fallacy.

The Hot Hand in the NBA 3-Point Contest: The Importance of Location, Location, Location

“…Our results tie together multiple themes from previous research into the hot hand hypothesis in basketball. First, our results align with those from controlled shooting experiments which find a hot hand, such as Miller & Sanjurjo (2018)'s paradigmshifting article on the hot hand and gambler's fallacy, despite the power issues pointed out by Ritzwoller & Romano (2022). Our finding that the existence of a hot hand for shots within the same location also provides support for previous papers which find a hot hand in NBA free throw shooting (e.g., Arkes, 2010;Lantis & Nesson, 2021).…”

“…After correcting for this bias, the original data in Gilovich et al (1985) support the existence of a hot hand. However, Ritzwoller & Romano (2022) develop new tests for evaluating the proportion of successes following consecutive successes with either the overall proportion of successes or the proportion of successes following consecutive failures. Applying their tests to the Gilovich et al (1985) controlled shooting experiment data, they find that the hot hand found by Miller & Sanjurjo (2018) is driven by only one shooter.…”

“…Previous studies identify a hot hand in free throws taken within the same set (e.g., Aharoni & Sarig, 2012;Arkes, 2010;Lantis & Nesson, 2021;Miller & Sanjurjo, 2021;Yaari and Eisenmann, 2011). Additional support for the existence of the hot hand is also found in controlled shooting experiments, although many of these experiments may be underpowered (Miller & Sanjurjo, 2014Ritzwoller & Romano, 2022). Evidence from the run of play, the most relevant setting to the belief in the hot hand, has been mixed (e.g., Bocskocsky et al, 2014;Bühren & Krabel, 2015;Csapo & Raab, 2014;Lantis & Nesson, 2021;Rao, 2009).…”

Permutation testing for dependence in time series

“…(2006) consider the application of permutation testing as a method of testing for periodicity in biological data. In a more theoretical setting, Jentsch and Pauly (2015) use randomization methods to test equality of spectral densities, and Ritzwoller and Romano (2020) consider permutation testing in the setting of dependent Bernoulli sequences.…”

Permutation Testing for Dependence in Time Series

“…In the context of time series data, Nichols and Holmes (2002) discuss the application of permutation testing to neuroimaging data, and Ptitsyn et al (2006) consider the application of permutation testing as a method of testing for periodicity in biological data. In a more theoretical setting, Jentsch and Pauly (2015) use randomization methods to test equality of spectral densities, and Ritzwoller and Romano (2020) consider permutation testing in the setting of dependent Bernoulli sequences.…”