Optimality of perceptual decision criteria.

Ulehla, Z. Joseph

doi:10.1037/h0023007

Cited by 62 publications

(75 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, a series ofprevious studies (Healy & Kubovy, 1981;Lee & Janke, 1965;Ulehla, 1966) suggests that conservative cutoffplacement is likely to result when category base-rates are unequal. Why is extreme cutoff placement so prevalent in one set of studies and conservative cutoffplacement so prevalent in another set of studies?…”

Section: Conservative Versus Extreme Cutoff Placementmentioning

confidence: 99%

“…Maddox outlined several differences between the two sets of studies that might account for the discrepant findings. In many cases, the previous studies (Healy & Kubovy, 1981;Lee & Janke, 1965;Ulehla, 1966) used fairly inexperienced observers, analyzed the data collapsed across observers, and used unidimensional stimuli. Maddox (1995;Maddox & Bohil, 1998) and the present study used highly experienced observers, analyzed the data at the individual observer level, and used twodimensional stimuli.…”

Section: Conservative Versus Extreme Cutoff Placementmentioning

confidence: 99%

“…Some of the early signal detection studies and several early categorization studies utilized such an approach (e.g., Green & Swets, 1966;Healy & Kubovy, 1981;Kubovy & Healy, 1977;Lee & Janke, 1964, 1965Lee & Zentall, 1966;Ulehla, 1966). In the categorization studies, the two categories were usually univariate normally distributed, and category base-rates were manipulated.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Overestimation of base-rate differences in complex perceptual categories

Maddox

Bohil

1998

Perception & Psychophysics

View full text Add to dashboard Cite

The optimality of multidimensional perceptual categorization performance was examined for several base-rate ratios, for both integral and separable dimension stimuli, and for complex category structures. In all cases, the optimal decision bound was highly nonlinear. Observers completed several experimental sessions, and all analyses were performed at the single-observer level using a series of nested models derived from decision-bound theory (Maddox, 1995;. In every condition, all observers were found to be sensitive to the base-rate manipulations, but the majority of observers appeared to overestimate the base-rate difference. These findings converge with those for cases in which the optimal decision bound was linear (Maddox, 1995) and suggest that base-rates are learned in a similar fashion regardless of the complexity of the optimal decision bound. Possible explanations for the consistent overestimate of the base-rate difference are discussed. Several continuous-valued analogues of Kruschke's (1996) theory of base-rate learning with discrete-valued stimuli were tested. These models found some support, but in all cases were outperformed by a version of decision-bound theory that assumed accurate knowledge of the category structure and an overestimate of the base-rate difference.Categorization is fundamental to human survival. Every day we make hundreds ofcategorization judgments, and in many cases are very accurate. For example, our ability to categorize speech sounds and handwritten characters is unmatched by even the most sophisticated machines. Thus, it is reasonable to suppose that in certain domains our categorization performance is very nearly optimal (Ashby & Maddox, 1998). A major goal of the present study was to examine the optimality of human categorization performance when the observer is faced with a complex categorization problem similar in spirit to speech sound or handwritten character categorization.To study optimality, one must rigorously define it. Although optimality can be defined in many ways, a reasonable definition, and the one we choose, is performance that maximizes long-run accuracy (or long-run reward; Ashby, 1992a;Green & Swets, 1966;Morrison, 1967). When the costs and benefits associated with each categorization response bias the observer toward one response or the other, performance that maximizes longrun accuracy may not maximize long-run reward. In the present study, cost and benefits were not manipulated, so the strategy that maximized long-run reward also maximized long-run accuracy. Maddox and Bohil (1998) ined the effects of cost-benefit manipulations on categorization performance and found that observers were sensitive to this type of information.The optimal classifier is a hypothetical device that integrates information in such a way as to maximize longrun accuracy. For present purposes, two sources of information are relevant to the optimal classifier. One source is information about the distribution ofcategory exemplarsthat is, information about the central tendency a...

show abstract

Section: Conservative Versus Extreme Cutoff Placementmentioning

confidence: 99%

Section: Conservative Versus Extreme Cutoff Placementmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Overestimation of base-rate differences in complex perceptual categories

Maddox

Bohil

1998

Perception & Psychophysics

View full text Add to dashboard Cite

show abstract

“…For example, if the benefit of a correct Category A response is 3 points, the benefit of a correct Category B response is 1 point, and the cost of an incorrect response is 0 points (referred to as a 3:1 zerocost condition, because no loss of points is associated with an incorrect response), the optimal classifier decision criterion, b o = 3, maximizes long-run reward. Under these conditions, observed decision criterion values fall somewhere between this reward-maximizing criterion (b o = 3) and the accuracy-maximizing criterion (b 5 1; see, e.g., Green & Swets, 1966;Healy & Kubovy, 1981;Kubovy & Healy, 1977;Lee & Janke, 1964, 1965Lee & Zentall, 1966;Ulehla, 1966). This is termed conservative cutoff placement, because the decision criterion is not shifted far enough toward the optimal value.…”

mentioning

confidence: 99%

On the generality of optimal versus objective classifier feedback effects on decision criterion learning in perceptual categorization

Bohil

Maddox

2003

Memory & Cognition

View full text Add to dashboard Cite

Biased category payoff matrices engender separate reward-and accuracy-maximizingdecision criteria. Although instructed to maximize reward, observers use suboptimal decision criteria that place greater emphasis on accuracy than is optimal. In this study, objective classifier feedback (the objectively correct response) was compared with optimal classifier feedback (the optimal classifier' s response) at two levels of category discriminability when zero or negative costs accompanied incorrect responses for two payoff matrix multiplication factors. Performance was superior for optimal classifier feedback relative to objective classifier feedback for both zero-and negative-cost conditions, especially when category discriminability was low, but the magnitude of the optimal classifier advantage was approximately equal for zero-and negative-cost conditions. The optimal classifier feedback performance advantage did not interact with the payoff matrix multiplication factor. Model-based analyses suggested that the weight placed on accuracy was reduced for optimal classifierfeedback relative to objective classifier feedback and for high category discriminability relative to low category discriminability. In addition, the weight placed on accuracy declined with training when feedback was based on the optimal classifier and remained relativelystable when feedback was based on the objective classifier.These results suggest that feedback based on the optimal classifier leads to superior decision criterion learning across a wide range of experimental conditions.

show abstract

“…The research described here included two tests of the integration model in a type of visual discrimination task previously used to investigate other aspects of SOT (Ulehla, 1966). This task involved the judgment of the direction of tilt (right vs left) of lines rotated Y2 deg from the vertical as defined by an enclosing square.…”

mentioning

confidence: 99%

Integration of information in a visual discrimination task

Ulehla

Halpern

Cerf

1968

Perception & Psychophysics

View full text Add to dashboard Cite

2The integration of information model ofsignal detection theory (SDT) was tested in a multiple observation tilt-discrimination task. Improvement in discriminability with additional observations was consonant with theory over two observations, but observations beyond the second yielded less improvement than predicted by the integration model. Analyses of operating characteristics lent some support to the applicability of SDr to the tilt-discrimination task.This paper describes two experiments designed to test the integration model of signal detection theory (SOT) in a multiple observation task (cf. Green & Swets, 1966). In opposition to traditional threshold theory which posits an all-or-nothing informing effect of stimuli, SOT states that the informing effects range along a continuum from determinate information favoring one stimulus alternative through no information to determinate information favoring the other alternative. A stimulus is evaluated by comparing its informing effect with an internal standard or criterion. The probability density function of the informing effects yielded by a given stimulus source is assumed to be normal. In a two-alternative discrimination task, the informing effects density functions associated with the alternative stimuli would differ in location, but, for the sake of simplicity, can be assumed to be equal in variance. The difference between the means of the probability density functions, in units of the density function standard deviation, is labeled the index of signal detectability (d') and constitutes the SOT measure of discriminability-sensitivity.In order to derive predictions of discriminability when multiple observations of the stimulus are furnished on each trial, it is assumed that the informing effect associated with a set of independent observations is the mean of the informing effects of the component observations. Thus, the probability density function for a set of n observations should possess the same location on the informing effects continuum as the density function for single observations. However, the standard deviation should be only If./ii (n being the number of observations) of the standard deviation of the single observation density function. Because d' is the ratio of the difference in location to the standard deviation of the density functions, reduction in the standard deviation yields a proportional increase in d'. Thus, the d' based on n observations per trial is predicted by the SOT integration model to be v'n times the d' based on one observation per trial. The v'nprediction requires the assumption that the multiple stimulus observations are mutually independent. If dependence should exist, the predicted improvement would be less than v'n. It may be noted that the same predictions can be derived if one assumes that S bases his judgment on the likelihood ratio of the informing effects, and that the likelihood ratio of a set of stimulus observations is the product of the likelihood ratios of the individual observations (Green & Swets, 1966).In an a...

show abstract

Optimality of perceptual decision criteria.

Cited by 62 publications

References 7 publications

Overestimation of base-rate differences in complex perceptual categories

Overestimation of base-rate differences in complex perceptual categories

On the generality of optimal versus objective classifier feedback effects on decision criterion learning in perceptual categorization

Integration of information in a visual discrimination task

Contact Info

Product

Resources

About