Omitted-Variable Bias and Other Matters in the Defense of the Category Adjustment Model: A Reply to Crawford (2019)

Abstract

The datasets from Duffy, Huttenlocher, Hedges, and Crawford (2010) [Psychonomic Bulletin & Review, 17(2), [224][225][226][227][228][229][230] were reanalyzed by Duffy and Smith (2018) [Psychonomic Bulletin & Review, 25(5), 1740-1750. Duffy and Smith (2018) conclude that the datasets are not consistent with the category adjustment model (CAM). Crawford (2019) [Psychonomic Bulletin & Review, 26(2), 693-698] offered a reply to Duffy and Smith (2018) that is based on three main points.Crawford proposes regressions that are, in part, based on a "deviation" analysis. Crawford offers a different simulation of data and claims that the techniques employed by Duffy and Smith (2018) are not sufficiently sensitive to detect a specific relationship that is claimed to be consistent with CAM. Crawford also appeals to a figure showing that the responses appear to be biased toward the overall running mean, and presumably not toward recently viewed lines. We show that Crawford's analysis suffers from an omitted-variable bias. Once this bias is corrected, the evidence in support of CAM disappears. When we produce a simulated dataset that is consistent with the specification suggested by Crawford, the techniques of Duffy and Smith (2018) correctly detect the true relationship. Despite the assertion otherwise, the simulated dataset that was analyzed by Crawford is not publicly available. Since the analysis of Crawford (2019) is incorrect, it remains our view that the datasets from Duffy, Huttenlocher, Hedges, and Crawford (2010) do not appear to be consistent with CAM or any Bayesian model of judgment.

Central tendency bias in belief elicitation

“…Holding the utility function constant, such shift of beliefs might be expected to lead to a higher bid. 26 To examine the extent to which this theoretical comparative static is borne in between-subject comparisons, we rank-correlate across subjects their bid with the probability implied by the response that the bid of the opponent does not exceed the upper boundary of Bin i, repeating the exercise for i ∈ {1, .., 4}. To make this exercise meaningful as a reflection of the comparative static discussed above, we must assume that subjects reporting lower probabilities up to Bin i are not systematically more risk averse than subjects reporting higher probabilities.…”

“…Duffy and Smith (2018) analyze the data from Duffy, Huttenlocher, Hedges, and Crawford (2010), and come to a similar conclusion about the predictions of the category adjustment model. See also Crawford (2019) and Duffy and Smith (2019b). 2 tendency bias because there appears to be such a bias toward the mean of the distribution.…”