Information-theoretic signal detection theory.

Abstract

Signal detection theory (SDT), the standard mathematical framework by which we understand how stimuli are classified into distributions such as signal or noise, is an essential part of the modern psychologist's toolkit. This article introduces some mathematical tools derived from information theory which allow surprisingly simple approximations to key quantities in SDT. The main idea is a lower bound on the probability of correct classification of a stimulus, as a function of information-theoretic properties of the generating distribution. This bound depends on three distinct factors, each of which can be quantified information-theoretically: (a) The prior uncertainty in the choice of generating distribution; (b) the inherent separability of the classes; and (c) the discrepancy between the observer's model of the class distributions and the "true" model. The bound is only a loose substitute for the conventional method for computing proportion correct (via integration) but generalizes readily to multiple dimensions and larger numbers of stimulus categories, where direct integration is computationally difficult. Moreover, unlike most conventional SDT formulae, this bound does not require Gaussian distributions. Most importantly, the information-theoretic signal detection theory (IT-SDT) framework sheds light on the way classification performance depends on the discrepancy between the observer's assumptions and those actually governing the environment.

Ready to detect a reversal of time's arrow: a psychophysical study using short video clips in daily scenes

“…Here, p(f|f) denotes the probability of the ‘forward’ response in response to ‘forward’ presentation. By applying signal detection theory [16] to p(f|f) and p(r|r), we calculated the level of separation ( d′ ) and response bias (positive for the bias favouring the forward judgement) by using the formula as follows:dnormal′=norminvfalse(pfalse( f|ffalse)false)+norminvfalse(pfalse(r|rfalse)false)andbias=false(norminvfalse( p( ffalsefalse|f)false)−norminvfalse( p(rfalsefalse|r)false)/2,where norminv denotes the normal inverse cumulative distribution function. We substituted 0.01 and 0.99 for 0 and 1, respectively, to prevent the values from diverging to infinity.…”

Probabilistic origins of compositional mental representations.

“…By definition, fidelity is maximal for the feature corresponding to the ideal observer and is less for other features, but again is only near 1 with a very well-separated mixture for which the overlap between components is negligible. The uncertainty that remains even for an ideal observer using all available features is the ineliminable uncertainty, attributable to the intrinsic overlap among the classes (see below and see Feldman, 2021a).…”

“…Even an ideal observer—that is, one who (a) possesses an accurate model of the environment and (b) uses it optimally to classify data—cannot necessarily answer this question accurately all the time. In most realistic mixtures, the component distributions overlap to some degree, meaning for that each datum there remains some ineliminable ambiguity about its origin (see Feldman, 2021a). However, a rational observer (including an ideal one) can divide the feature space into regions within which each hypothetical class has maximum-posterior probability.…”

“…Fidelity is capped at 1 because I ( K ; X ) cannot exceed H ( K ), but as will become clear below, is generally less than 1 for realistic mixtures. That is, for all but the most extremely “well-separated” mixtures (i.e., those whose mixture components have variances that are very small compared to the distance between their means), some intrinsic uncertainty about the likely origins of each datum will remain, and Q will be less than 1 (see Feldman, 2021a). Note that the term “intrinsic” here means that this uncertainty does not stem from the limitations of a small sample; even when the distribution is known precisely (i.e., even asymptotically and even for an ideal observer) data drawn from p cannot be classified unambiguously if fidelity is less than 1.…”

“…5As discussed in Feldman (2021a), generally proportion correct in classifying a variable K (here, the true mixture component) based on a derived quantity Z (here, the mixture component estimated from the projection) is bound by 2−Hfalse(Kfalse|zfalse), in which the negative exponent is the uncertainty in the posterior p(K|Z). Because I(K;Z)=H(K)−H(K|Z), so H(K|Z)=H(K)−I(K;Z), and Q=I(K;Z)/H(K), the performance bound can be written as 2false(Q−1false)Hfalse(Kfalse).…”