Reply to Duffy and Smith’s (2018) reexamination

Abstract

Duffy, Huttenlocher, Hedges, and Crawford (2010, Psychonomic Bulletin & Review, 17[2], 224-230) examined whether the well-established central tendency bias in people's reproductions of stimuli reflects bias toward the mean of an entire presented distribution or bias toward only recently seen stimuli. They reported evidence that responses were biased toward the long-run mean and found no evidence that they were biased toward the most recent stimuli. Duffy and Smith (2018) reexamine the data using a different analytical strategy and argue that estimates are biased by recent stimuli rather than toward the long-run mean. I argue that this reanalysis misses a true effect of the running mean and that the data are (mostly) consistent with the claims in the original work. I suggest that these results, and many other null results presented by Duffy and Smith, do not have major theoretical significance for the category adjustment model and similar Bayesian models. (Code and data available: https://osf. io/tkqvn.) Keywords Human memory . Statistical inference . CategorizationAt issue in earlier work by Duffy, Huttenlocher, Hedges, and Crawford (2010, hereafter called DHHC) was whether the well-established central tendency bias in people's reproductions of stimuli reflects bias toward the mean of an entire distribution or bias toward recently seen stimuli. DHHC reported evidence that responses were biased toward the long-run mean rather than toward the most recent stimuli and presented these results as consistent with the category adjustment model (CAM), a Bayesian model of stimulus estimation (Huttenlocher, Hedges, & Vevea, 2000). In their reexamination, Duffy and Smith (DS; 2018) point out some flaws in the original work, apply a different analytical approach to the data, and conclude that estimates are biased toward recent stimuli rather than toward the running mean. They interpret their findings as a refutation of CAM, and of Bayesian models more generally. Here, I argue that the DS failure to detect the impact of the running mean on estimates is likely a Type II error. I also suggest that the paper overstates the theoretical significance of its findings and of the original findings reported in DHHC, and that it presents a distorted view of CAM.

Omitted-Variable Bias and Other Matters in the Defense of the Category Adjustment Model: A Reply to Crawford (2019)

“…The case for including Target in the analysis is that it has been known to be significantly related to Response Bias for quite some time (Hollingworth, 1910 (2) is not valid and any inferences based on that analysis are likely to be incorrect. In summary, we reject the deviation analysis offered by Crawford (2019), which clearly suffers from an omitted-variable bias.…”

“…-2 Log L refers to negative two times the loglikelihood. Table 4 corresponds to Crawford's interpretation of Figure 1 in Crawford (2019): there is a significant and negative relationship between Target and Response Bias in both halves, but the Large Mean variable is only significant in the first half of trials.…”

“…Second, Crawford offers a different simulation of data and claims that the techniques employed in DS are not sufficiently sensitive to detect the relationship in support of CAM. When we produce a simulated dataset 17 consistent with the specification in Crawford (2019) our methods correctly identify the true relationship.…”

“…12 Footnote 10 in DS. 13 Crawford (2019) reports a specification of "Response = 1.8*Target + 0.1*Running Mean + 0.1*Last3 + ε" although we suspect that the actual coefficient of Target is 0.8, not 1.8. 14 Crawford (2019) does not report the standard deviation, but from the code it seems to be 25.…”

“…This journal recently published a reply to DS. In it, Crawford (2019) offers a defense of DHHC, in particular, and CAM, in general. The reply is based on three main points.…”