A general rule for sensory cue summation: evidence from photographic, musical, phonetic and cross-modal stimuli

Abstract

The Euclidean and MAX metrics have been widely used to model cue summation psychophysically and computationally. Both rules happen to be special cases of a more general Minkowski summation rule ðCue, where m ¼ 2 and 1, respectively. In vision research, Minkowski summation with power m ¼ 3 -4 has been shown to be a superior model of how subthreshold components sum to give an overall detection threshold. Recently, we have previously reported that Minkowski summation with power m ¼ 2.84 accurately models summation of suprathreshold visual cues in photographs. In four suprathreshold discrimination experiments, we confirm the previous findings with new visual stimuli and extend the applicability of this rule to cue combination in auditory stimuli (musical sequences and phonetic utterances, where m ¼ 2.95 and 2.54, respectively) and cross-modal stimuli (m ¼ 2.56). In all cases, Minkowski summation with power m ¼ 2.5 -3 outperforms the Euclidean and MAX operator models. We propose that this reflects the summation of neuronal responses that are not entirely independent but which show some correlation in their magnitudes. Our findings are consistent with electrophysiological research that demonstrates signal correlations (r ¼ 0.1 -0.2) between sensory neurons when these are presented with natural stimuli.

The Role of Binocular Disparity in Rapid Scene and Pattern Recognition

“…Furthermore, the question of how disparity and luminance contribute to pattern recognition is formally related to the question of how different sources of information contribute to the detection of threshold stimuli, and specifically of whether subthreshold summation takes place (e.g. Meinhardt, Persike, Mesenholl, & Hagemann, 2006), and to the question of how changes in individual dimensions contribute to the appearance of complex stimuli (Landy et al, 1995; To, Baddeley, Troscianko, & Tolhurst, 2011; To, Lovell, Troscianko, & Tolhurst, 2008). In all of these contexts, it is possible to compare the performance which is observed when combined stimuli are presented with the performance which can be predicted based on the results of the single-modality stimuli assuming independent processing of the two dimensions and linear combination of the responses.…”

“…To et al, 2011). Other models such as Euclidean summation (Minkowski exponent equal to 2) or MAX rule summation (infinite Minkowski exponent) predict a lesser increase of performance when both stimuli are presented at the same time.…”

Processing bimodal stimuli: integrality/separability of color and orientation

“…A minimum of 0.47 is achieved at p = 1.30, compared to the values of 1.02 and 0.99 at p = 1 and 2, respectively. Note that RMSE is closely related to the summed residuals used by To et al (2008, 2011) and the Pearson correlation r used by Shepard and Cermak (1973) (see also Dunn, 1983). Predicted dissimilarities for p = 1.3 are plotted against observations in Figure 8D.…”

“…Shepard's view (1987, Figure 4) of consequential neighborhoods makes the same prediction. For small enough differences between stimuli (or between stimulus and background), there is a threshold of discrimination where the detection of any change is limited by the specific sensory channel on which the difference is greatest (To et al, 2011). The contribution from any sub-threshold differences coded on other channels is small (in the case of probabilistic detection models) or zero.…”

“…To et al (2008, 2011) proposed that a large p can be expected when the variations along the dimensions are strongly correlated, exhibiting a high level of redundancy, so that a stimulus difference along one attribute is normally accompanied by a comparable difference on the others. In informational terms, such differences can be encoded efficiently if the neural nexus at which they converge allows the larger of the signals to dominate others that add little or no further information about dissimilarity.…”