Rarity, pseudodiagnosticity and Bayesian reasoning

Abstract

Three experiments investigated the effect of rarity on people's selection and interpretation of data in a variant of the pseudodiagnosticity task. For familiar (Experiment 1) but not for arbitrary (Experiment 3) materials, participants were more likely to select evidence so as to complete a likelihood ratio when the initial evidence they received was a single likelihood concerning a rare feature. This rarity effect with familiar materials was replicated in Experiment 2 where it was shown that participants were relatively insensitive to explicit manipulations of the likely diagnosticity of rare evidence. In contrast to the effects for data selection, there was an effect of rarity on confidence ratings after receipt of a single likelihood for arbitrary but not for familiar materials. It is suggested that selecting diagnostic evidence necessitates explicit consideration of the alternative hypothesis and that consideration of the possible consequences of the evidence for the alternative weakens the rarity effect in confidence ratings. Paradoxically, although rarity effects in evidence selection and confidence ratings are in the spirit of Bayesian reasoning, the effect on confidence ratings appears to rely on participants thinking less about the alternative hypothesis.When interpreting evidence (D) with which to decide between a pair of hypotheses, X and Y, we need to consider both the prior probabilities of the hypotheses (P(Hx) and P(Hy)), and the probability of the evidence under Correspondence should be addressed to

Relevance-Driven Information Search in “Pseudodiagnostic” Reasoning

“…By contrast, Feeney, Evans, and Venn (2008) showed that simply offering anchoring statistical information about a rare feature was enough to increase the rate of row choices. Our results corroborate and extend Feeney et al's findings and therefore also contrast with Evans et al's (2002) assumption by demonstrating that, once the relevance constraint is satisfied, reasoners can engage in diagnostic reasoning comparing values associated with a unique feature across alternative hypotheses even in the absence of explicit instructions to consider the alternative option.…”

Missing the dog that failed to bark in the nighttime: on the overestimation of occurrences over non-occurrences in hypothesis testing

“…The influence of the rarity of the absent clues Rarity effects concern the apportionment of increased attention to rare events in contrast to common ones (e.g., Feeney, Evans & Clibbens, 2000;Feeney, Evans & Venn, 2008;Green & Over, 2000;McKenzie & Mikkelsen, 2000Oaksford & Chater, 1994;2003;in legal contexts, for example, see Loftus, 1976;Wells & Lindsay, 1980). We included in our initial predictions a hypothesis that was based on rarity effects, conjecturing that participants would possibly pay more heed to absent clues when they were rare in comparison to present ones.…”

Pseudodiagnosticity revisited.

“…The above claims are supported by the following remarks. To begin with, the rare versus common feature manipulation along with the use of real-life contents produces departures from the uniformity assumption, as suggested by Feeney et al's (2008) manipulation check. For instance, in the "common" condition from their Experiment 2, Feeney et al identified average expected row values as varying from .49 to up to .92 across four problem contents, with average expected column/diagonal values concurrently ranging between .56 and .84.…”