Overestimation of base-rate differences in complex perceptual categories

Abstract

The optimality of multidimensional perceptual categorization performance was examined for several base-rate ratios, for both integral and separable dimension stimuli, and for complex category structures. In all cases, the optimal decision bound was highly nonlinear. Observers completed several experimental sessions, and all analyses were performed at the single-observer level using a series of nested models derived from decision-bound theory (Maddox, 1995;. In every condition, all observers were found to be sensitive to the base-rate manipulations, but the majority of observers appeared to overestimate the base-rate difference. These findings converge with those for cases in which the optimal decision bound was linear (Maddox, 1995) and suggest that base-rates are learned in a similar fashion regardless of the complexity of the optimal decision bound. Possible explanations for the consistent overestimate of the base-rate difference are discussed. Several continuous-valued analogues of Kruschke's (1996) theory of base-rate learning with discrete-valued stimuli were tested. These models found some support, but in all cases were outperformed by a version of decision-bound theory that assumed accurate knowledge of the category structure and an overestimate of the base-rate difference.Categorization is fundamental to human survival. Every day we make hundreds ofcategorization judgments, and in many cases are very accurate. For example, our ability to categorize speech sounds and handwritten characters is unmatched by even the most sophisticated machines. Thus, it is reasonable to suppose that in certain domains our categorization performance is very nearly optimal (Ashby & Maddox, 1998). A major goal of the present study was to examine the optimality of human categorization performance when the observer is faced with a complex categorization problem similar in spirit to speech sound or handwritten character categorization.To study optimality, one must rigorously define it. Although optimality can be defined in many ways, a reasonable definition, and the one we choose, is performance that maximizes long-run accuracy (or long-run reward; Ashby, 1992a;Green & Swets, 1966;Morrison, 1967). When the costs and benefits associated with each categorization response bias the observer toward one response or the other, performance that maximizes longrun accuracy may not maximize long-run reward. In the present study, cost and benefits were not manipulated, so the strategy that maximized long-run reward also maximized long-run accuracy. Maddox and Bohil (1998) ined the effects of cost-benefit manipulations on categorization performance and found that observers were sensitive to this type of information.The optimal classifier is a hypothetical device that integrates information in such a way as to maximize longrun accuracy. For present purposes, two sources of information are relevant to the optimal classifier. One source is information about the distribution ofcategory exemplarsthat is, information about the central tendency a...

On the relation between decision rules and perceptual representation in multidimensional perceptual categorization

“…The fact that categorization performance is strongly affected by the linearity of the categorization rule makes this an important hypothesis to test (see Ashby & Gott, 1988;Ashby & Maddox, 1990, 1992Maddox, Ashby, & Gottlob, 1998;Maddox & Bohil, 1998; see also Brehmer, 1987, Hammond & Summers, 1972, and Mellers, 1980, for similar findings from the multiple cue probability learning literature; but see Medin & Schwanenflugel, 1981). Specifically, we estimated one pair of additive scalars (one for orientation and one for length) that were applied to the nonlinear categorization conditions (i.e., crisscross and interior-exterior) and another pair of additive scalars that were applied to the linear categorization conditions (i.e., linear integration and decisional selective attention).…”

Category discriminability, base-rate, and payoff effects in perceptual categorization

“…The perceptual and cognitive processes involved in solving categorization problems of this sort have been studied extensively (Ashby, 1992a;Busemeyer & Myung, 1992;Green & Swets, 1966;Healy & Kubovy, 1981;Koehler, 1996;Macmillan & Creelman, 1991;Maddox, 1995;Maddox & Ashby, 1993;Maddox & Bohil, 1998a, 1998bStevenson, Busemeyer, & Naylor, 1991;von Winterfeldt & Edwards, 1982). A fruitful approach has been to compare human performance with that of the optimal classifier-a hypothetical device whose categorization decisions maximize long-run reward.…”

“…This approach is useful because the effects of category discriminability, base rates, and payoffs are identifiable in the optimal classifier's performance and thus can be investigated systematically. Several studies (to be reviewed shortly) have examined base-rate and payoff sensitivity in perceptual categorization tasks, although most of these have examined either base-rate or payoff manipulations (Busemeyer & Myung, 1992;Lee & Janke, 1964, 1965361 Copyright 2001 Psychonomic Society, Inc. Maddox, 1995;Maddox & Bohil, 1998b) and have not manipulated both within the same experimental framework (however, see Green & Swets, 1966;Healy & Kubovy, 1981;Maddox & Bohil, 1998a). To our knowledge, no studies have examined the effects of category discriminability on base-rate and payoff sensitivity in perceptual categorization (however, see Kubovy & Healy, 1980).…”

Independence and separability in the perception of complex nonspeech sounds

“…Another way to generate data with degrees of freedom sufficient to constrain the general Gaussian GRT model is to manipulate stimulus presentation base rates (and/ or payoff schemes; see, e.g., Maddox, 1995;Maddox & Bohil, 1998a, 1998b, 2003 to produce, for example, 4 4 confusion matrices for multiple experimental conditions. It is worth noting that, although base-rate manipulations can produce data sufficiently rich to constrain the GRT model, an additional assumption about how (or whether) perceptual distributions and decision bounds change across base-rate conditions is required.…”