Modelling auditory attention

Kaya, Emine Merve; Elhilali, Mounya

doi:10.1098/rstb.2016.0101

Cited by 109 publications

(111 citation statements)

References 87 publications

Supporting

Mentioning

106

Contrasting

Unclassified

Order By: Relevance

“…This promises new insight on the much-debated delineation of functional sub-regions of auditory cortex and adds to the demonstrated utility of neural networks in parcellating not only speech but also the gist of auditory scenes (e.g. [46]). Fortunately, excellent software toolboxes to aid in DNN training are readily available [47,48].…”

Section: A Tripartite Approach To Tackle Current Methodological Challmentioning

confidence: 99%

Resolving the neural dynamics of visual and auditory scene processing in the human brain: a methodological approach

Cichy

Teng

2017

Phil. Trans. R. Soc. B

View full text Add to dashboard Cite

In natural environments, visual and auditory stimulation elicit responses across a large set of brain regions in a fraction of a second, yielding representations of the multimodal scene and its properties. The rapid and complex neural dynamics underlying visual and auditory information processing pose major challenges to human cognitive neuroscience. Brain signals measured non-invasively are inherently noisy, the format of neural representations is unknown, and transformations between representations are complex and often nonlinear. Further, no single non-invasive brain measurement technique provides a spatio-temporally integrated view. In this opinion piece, we argue that progress can be made by a concerted effort based on three pillars of recent methodological development: (i) sensitive analysis techniques such as decoding and cross-classification, (ii) complex computational modelling using models such as deep neural networks, and (iii) integration across imaging methods (magnetoencephalography/electroencephalography, functional magnetic resonance imaging) and models, e.g. using representational similarity analysis. We showcase two recent efforts that have been undertaken in this spirit and provide novel results about visual and auditory scene analysis. Finally, we discuss the limits of this perspective and sketch a concrete roadmap for future research.This article is part of the themed issue ‘Auditory and visual scene analysis’.

show abstract

Section: A Tripartite Approach To Tackle Current Methodological Challmentioning

confidence: 99%

Resolving the neural dynamics of visual and auditory scene processing in the human brain: a methodological approach

Cichy

Teng

2017

Phil. Trans. R. Soc. B

View full text Add to dashboard Cite

show abstract

“…Similar to how ‘colour’ is used to compute salience in the visual modality, Southwell et al [6] have found that one salient property of auditory stimuli is ‘predictable repetition over time’ . For a broader background, this issue also includes comprehensive review comparing models of auditory and visual salience [7]. …”

Section: Visual Scene Analysis In the Brainmentioning

confidence: 99%

How is visual salience computed in the brain? Insights from behaviour, neurobiology and modelling

Veale

Hafed²,

Yoshida

2017

Phil. Trans. R. Soc. B

145

143

View full text Add to dashboard Cite

Inherent in visual scene analysis is a bottleneck associated with the need to sequentially sample locations with foveating eye movements. The concept of a ‘saliency map’ topographically encoding stimulus conspicuity over the visual scene has proven to be an efficient predictor of eye movements. Our work reviews insights into the neurobiological implementation of visual salience computation. We start by summarizing the role that different visual brain areas play in salience computation, whether at the level of feature analysis for bottom-up salience or at the level of goal-directed priority maps for output behaviour. We then delve into how a subcortical structure, the superior colliculus (SC), participates in salience computation. The SC represents a visual saliency map via a centre-surround inhibition mechanism in the superficial layers, which feeds into priority selection mechanisms in the deeper layers, thereby affecting saccadic and microsaccadic eye movements. Lateral interactions in the local SC circuit are particularly important for controlling active populations of neurons. This, in turn, might help explain long-range effects, such as those of peripheral cues on tiny microsaccades. Finally, we show how a combination of in vitro neurophysiology and large-scale computational modelling is able to clarify how salience computation is implemented in the local circuit of the SC.This article is part of the themed issue ‘Auditory and visual scene analysis’.

show abstract

“…In turn, theoretical frameworks developed to model empirical data remain limited in scope and fail to generalize beyond the contexts for which they were developed. 14 The current study aims to break this barrier and offer a baseline database for auditory salience from an unconstrained choice of stimuli. This collection includes a variety of complex natural scenes, which were selected with a goal of representing as many types of real world scenarios as possible.…”

mentioning

confidence: 99%

Auditory salience using natural soundscapes

Huang

Elhilali

2017

The Journal of the Acoustical Society of America

Self Cite

View full text Add to dashboard Cite

Salience describes the phenomenon by which an object stands out from a scene. While its underlying processes are extensively studied in vision, mechanisms of auditory salience remain largely unknown. Previous studies have used well-controlled auditory scenes to shed light on some of the acoustic attributes that drive the salience of sound events. Unfortunately, the use of constrained stimuli in addition to a lack of well-established benchmarks of salience judgments hampers the development of comprehensive theories of sensory-driven auditory attention. The present study explores auditory salience in a set of dynamic natural scenes. A behavioral measure of salience is collected by having human volunteers listen to two concurrent scenes and indicate continuously which one attracts their attention. By using natural scenes, the study takes a data-driven rather than experimenter-driven approach to exploring the parameters of auditory salience. The findings indicate that the space of auditory salience is multidimensional (spanning loudness, pitch, spectral shape, as well as other acoustic attributes), nonlinear and highly context-dependent. Importantly, the results indicate that contextual information about the entire scene over both short and long scales needs to be considered in order to properly account for perceptual judgments of salience.

show abstract

Modelling auditory attention

Cited by 109 publications

References 87 publications

Resolving the neural dynamics of visual and auditory scene processing in the human brain: a methodological approach

Resolving the neural dynamics of visual and auditory scene processing in the human brain: a methodological approach

How is visual salience computed in the brain? Insights from behaviour, neurobiology and modelling

Auditory salience using natural soundscapes

Contact Info

Product

Resources

About