Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Numbers

Abstract

We prove that a subtle but substantial bias exists in a common measure of the conditional dependence of present outcomes on streaks of past outcomes in sequential data. The magnitude of this streak selection bias generally decreases as the sequence gets longer, but increases in streak length, and remains substantial for a range of sequence lengths often used in empirical work. We observe that the canonical study in the influential hot hand fallacy literature, along with replications, are vulnerable to the bias. Upon correcting for the bias, we find that the longstanding conclusions of the canonical study are reversed.

Mouse tracking reveals structure knowledge in the absence of model-based choice

“…Thus, an inability to represent a stable transition structure may be at the root of the problem. This could be another manifestation of the hot-hand fallacy 39 .…”

Do (Australian) jockeys have hot hands?

“…Controlling for both wins and rides means that the higher strike rate of the jockey in the hot hand state (if it exists) will be reflected in more winning streaks rather than more wins. 11 This is the point made by Miller and Sanjurjo (2018). 12 Note that the ratio of the two variables and the sign of the difference is preserved under this normalisation as = and − = − 1000.…”

“…The results from a follow up meta-analysis (Avugos, et al 2012) provided sufficient evidence for the authors to find against the existence of hot hands in sport in general. However, Miller and Sanjurjo (2018) have recently uncovered a bias in the common measure of the conditional dependence of present outcomes on streaks of past outcomes in sequential data used in many studies. Once this bias is corrected for some of the negative conclusions from the earlier studies are reversed.…”

A Bridge from Monty Hall to the Hot Hand: The Principle of Restricted Choice

“…When confronted with the scientific evidence against their beliefs, even professional players and coaches were left unpersuaded, leading the hot hand to become known as a "massive and widespread cognitive illusion" (Kahneman 2011). 2 However, with the recent discovery of a surprising statistical bias (Miller and Sanjurjo 2018), it appears that the basketball community may have been right all along. In particular, to estimate a player's probability of making a shot, conditional on having made several in a row, Gilovich, Vallone, and Tversky (1985) and subsequent studies (1) selected the shot attempts that immediately followed a streak of several made shots (for example, three) and then (2) calculated the player's shooting percentage on these shots.…”