Revealing multisensory benefit with diffusion modeling

Murray, Carolyn A.; Larrea-Mancera, E. Sebastian Lelo de; Glicksohn, Arit; Shams, Ladan; Seitz, Aaron R.

doi:10.1016/j.jmp.2020.102449

Cited by 7 publications

(6 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, HDDM was able to reveal a benefit of multisensory stimuli in both visual and auditory performances. This finding is consistent with our previous findings showing that drift diffusion modeling provides a more sensitive measure of multisensory integration benefits (Murray et al (2020)).…”

Section: Discussionsupporting

confidence: 93%

“…However, some groups have recently proposed variants of drift diffusion models that reduce this large sample size requirement to a more manageable number through methods such as iterative simplex minimization (Gómez et al (2007); Leite and Ratcliff (2010); Van Zandt et al (2000)) or maximum likelihood estimation of fitted parameters (Drugowitsch et al (2014)). Hierarchical drift diffusion models (HDDMs) are one such example, and have recently shown to be able to detect multisensory integration in detection and discrimination tasks where accuracy, reaction time, and sensitivity index (d') failed to detect integration (Murray et al (2020)). This class of models utilizes prior distributions for the DDM parameters which provide a full posterior of each resulting parameter estimate while reducing the sample 2/10 size needed for a convergent, stable solution.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hierarchical drift diffusion modeling uncovers multisensory benefit in numerosity discrimination tasks

CHAU¹,

Murray²,

Shams³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Studies of accuracy and reaction time in decision making often observe a speed-accuracy tradeoff, where either accuracy or reaction time is sacrificed for the other. While this effect may mask certain multisensory benefits in performance when accuracy and reaction time are separately measured, drift diffusion models (DDMs) are able to consider both simultaneously. However, drift diffusion models are often limited by large sample size requirements for reliable parameter estimation. One solution to this restriction is the use of hierarchical Bayesian estimation for DDM parameters. Here, we utilize hierarchical drift diffusion models (HDDMs) to reveal a multisensory advantage in auditory-visual numerosity discrimination tasks. By fitting this model with a modestly sized dataset, we also demonstrate that large sample sizes are not necessary for reliable parameter estimation.

show abstract

Section: Discussionsupporting

confidence: 93%

Section: Introductionmentioning

confidence: 99%

Hierarchical drift diffusion modeling uncovers multisensory benefit in numerosity discrimination tasks

CHAU¹,

Murray²,

Shams³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the absence of data from such structures, some of which may serve as neural integrators of sensory evidence, we modeled the evidence accumulation process using a driftdiffusion model (DDM). The DDM has been used to formalize assumptions about the evidence accumulation and decision-making process across many different psychological paradigms (de Gee et al, 2020;Liu et al, 2015;Murray et al, 2020;Ratcliff and Murdock, 1976;Ratcliff et al, 2018;Tsunada et al, 2015Tsunada et al, , 2019, and psychoacoustic studies have modeled AM perception at the computational level (e.g. Dau et al, 1997), but connections have not been made between AM perception and the DDM.…”

Section: Effects Of Durationmentioning

confidence: 99%

Psychometric and subcortical neurometric measures of temporal discrimination in rhesus macaques

Mackey

Hauser

Schoenhaut

et al. 2022

Preprint

View full text Add to dashboard Cite

Temporal envelope fluctuations are abundant in nature and are critical for perception of complex sounds. While psychophysical sinusoidal amplitude modulation (SAM) processing studies have characterized the perception of SAM, and neurophysiological studies report a subcortical transformation from temporal to rate-based code, no studies have characterized this transformation in unanesthetized animals or in nonhuman primates. To address this, we recorded single-unit responses and compared derived neurometric measures in the cochlear nucleus (CN) and inferior colliculus (IC) to psychometric measures of modulation frequency (MF) discrimination in macaques. IC and CN neurons often exhibited tuned responses to SAM in their rate and spike-timing. Neurometric thresholds spanned a large range (2-200 Hz Δ MF). The lowest 40% of IC thresholds were less than or equal to psychometric thresholds, regardless of which code was used, while CN thresholds were greater than psychometric thresholds. Discrimination at 10-20 Hz could be explained by indiscriminately pooling 30 units in either structure, while discrimination at higher MFs was best explained by more selective pooling. This suggests that pooled brainstem activity was sufficient for AM discrimination. Psychometric and neurometric thresholds decreased as a function of stimulus duration, but IC and CN thresholds were greater and more variable than behavior at durations less than 500 ms. This slower subcortical temporal integration compared to behavior was consistent with a drift diffusion model which reproduced individual differences in performance and can constrain future neurophysiological studies of temporal integration. These measures provide an account of AM perception at the neurophysiological, computational, and behavioral levels.

show abstract

“…Participants were able to identify and localize the target numeral most rapidly, when congruent audiovisual speech was present. Consistently, congruent audiovisual speech resulted in higher drift rates, indicating that the complementary visual information did in fact aid the evidence accumulation process, and suggesting that drift rate presents a sensitive and reliable measure of multisensory benefits (Chau et al, 2021;Murray et al, 2020). Overall, the behavioral results are consistent with the well-established audiovisual facilitation effect, demonstrating that bimodal stimulus presentation facilitates recognition accuracy, speeds up response times (Molholm et al 2004, Giard & Peronnet, 1999, Miller, 1982, and improves perceptual sensitivity (Wahn et al 2017).…”

Section: Visual Speech In the Periphery Facilitates Sound Localizatio...mentioning

confidence: 84%

Cross-modal interactions at the audiovisual cocktail-party revealed by behavior, ERPs, and neural oscillations

Klatt

Begau

Schneider

et al. 2022

Preprint

View full text Add to dashboard Cite

Theories of attention argue that objects are the units of attentional selection. In real-word environments such objects can contain visual and auditory features. To understand how mechanisms of selective attention operate in multisensory environments, we created an audiovisual cocktail-party situation, in which two speakers (left and right of fixation) simultaneously articulated brief numerals. In three separate blocks, informative auditory speech was presented (a) alone or paired with (b) congruent or (c) uninformative visual speech. In all blocks, subjects localized a pre-defined numeral. While audiovisual-congruent and uninformative speech improved response times and speed of information uptake according to diffusion modeling, an ERP analysis revealed that this did not coincide with enhanced attentional engagement. Yet, consistent with object-based attentional selection, the deployment of auditory spatial attention (N2ac) was accompanied by visuo-spatial attentional orienting (N2pc) irrespective of the informational content of visual speech. Notably, an N2pc component was absent in the auditory-only condition, demonstrating that a sound-induced shift of visuo-spatial attention relies on the availability of audio-visual features evolving coherently in time. Additional analyses revealed cross-modal interactions in working memory and modulations of cognitive control. The preregistered methods and hypotheses of this study can be found at https://osf.io/vh38g.

show abstract

Revealing multisensory benefit with diffusion modeling

Cited by 7 publications

References 47 publications

Hierarchical drift diffusion modeling uncovers multisensory benefit in numerosity discrimination tasks

Hierarchical drift diffusion modeling uncovers multisensory benefit in numerosity discrimination tasks

Psychometric and subcortical neurometric measures of temporal discrimination in rhesus macaques

Cross-modal interactions at the audiovisual cocktail-party revealed by behavior, ERPs, and neural oscillations

Contact Info

Product

Resources

About