A temporal saliency map for modeling auditory attention

Kaya, Emine Merve; Elhilali, Mounya

doi:10.1109/ciss.2012.6310945

Cited by 28 publications

(19 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the first models to address this problem computed a temporal salience map similarly to the model of Kayser et al ., but considered all of the features as evolving temporally, rather than as two-dimensional images [37]. The feature space was expanded to include perceptual properties of sound: loudness, pitch and timbre.…”

Section: Models Of Auditory Attentionmentioning

confidence: 99%

Modelling auditory attention

Kaya¹,

Elhilali²

2017

Phil. Trans. R. Soc. B

Self Cite

109

101

View full text Add to dashboard Cite

Sounds in everyday life seldom appear in isolation. Both humans and machines are constantly flooded with a cacophony of sounds that need to be sorted through and scoured for relevant information—a phenomenon referred to as the ‘cocktail party problem’. A key component in parsing acoustic scenes is the role of attention, which mediates perception and behaviour by focusing both sensory and cognitive resources on pertinent information in the stimulus space. The current article provides a review of modelling studies of auditory attention. The review highlights how the term attention refers to a multitude of behavioural and cognitive processes that can shape sensory processing. Attention can be modulated by ‘bottom-up’ sensory-driven factors, as well as ‘top-down’ task-specific goals, expectations and learned schemas. Essentially, it acts as a selection process or processes that focus both sensory and cognitive resources on the most relevant events in the soundscape; with relevance being dictated by the stimulus itself (e.g. a loud explosion) or by a task at hand (e.g. listen to announcements in a busy airport). Recent computational models of auditory attention provide key insights into its role in facilitating perception in cluttered auditory scenes.This article is part of the themed issue ‘Auditory and visual scene analysis’.

show abstract

Section: Models Of Auditory Attentionmentioning

confidence: 99%

Modelling auditory attention

Kaya¹,

Elhilali²

2017

Phil. Trans. R. Soc. B

Self Cite

109

101

View full text Add to dashboard Cite

show abstract

“…Similar to visual saliency models (see section 3.2), auditory saliency models are built by abstracting features (intensity, frequency contrast, and temporal contrast) from the sound "intensity image, " which is a visual conversion of auditory time-frequency spectrograms and normalized to be an integrated saliency map (Kayser et al, 2005;Kalinli and Narayanan, 2007) (see Figure 2B). Considering that humans and other primate animals can process the pure auditory signals without any visual conversion, Kaya and Elhilali (2012) modify the auditory saliency model by directly extracting the multi-dimensional temporal auditory signal features (envelope, frequency, rate, bandwidth, and pitch) of the auditory scene as input. Their model relies on the selection of parameters to reduce error rates of the saliency determination by fewer features.…”

Section: Computational Models For the Human Cocktail Party Problem Somentioning

confidence: 99%

What Can Computational Models Learn From Human Selective Attention? A Review From an Audiovisual Unimodal and Crossmodal Perspective

Weber

Yang

et al. 2020

Front. Integr. Neurosci.

View full text Add to dashboard Cite

Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments and computational modeling. Although an increasing number of findings on crossmodal selective attention have shed light on humans' behavioral patterns and neural underpinnings, a much better understanding is still necessary to yield the same benefit for intelligent computational agents. This article reviews studies of selective attention in unimodal visual and auditory and crossmodal audiovisual setups from the multidisciplinary perspectives of psychology and cognitive neuroscience, and evaluates different ways to simulate analogous mechanisms in computational models and robotics. We discuss the gaps between these fields in this interdisciplinary review and provide insights about how to use psychological findings and theories in artificial intelligence from different perspectives.

show abstract

“…The nature of sound as a time-evolving entity cannot be captured by spatial processing. There have been attempts to remedy this problem by changes to the procedure of computing saliency after feature extraction, but the methodologies used are still adaptations from vision mechanisms (Kaya and Elhilali, 2012 ; Cottrell and Tsuchida, 2012 ). In this work, we discard the traditional framework of computing a spatial saliency map, and employ psychoacoustical experimentation and computational modeling to build a saliency extraction mechanism that broadly mimics processes that are hyphothesized to take place in the auditory pathway.…”

Section: Introductionmentioning

confidence: 99%

Investigating bottom-up auditory attention

Kaya

Elhilali

2014

Front. Hum. Neurosci.

Self Cite

View full text Add to dashboard Cite

Bottom-up attention is a sensory-driven selection mechanism that directs perception toward a subset of the stimulus that is considered salient, or attention-grabbing. Most studies of bottom-up auditory attention have adapted frameworks similar to visual attention models whereby local or global “contrast” is a central concept in defining salient elements in a scene. In the current study, we take a more fundamental approach to modeling auditory attention; providing the first examination of the space of auditory saliency spanning pitch, intensity and timbre; and shedding light on complex interactions among these features. Informed by psychoacoustic results, we develop a computational model of auditory saliency implementing a novel attentional framework, guided by processes hypothesized to take place in the auditory pathway. In particular, the model tests the hypothesis that perception tracks the evolution of sound events in a multidimensional feature space, and flags any deviation from background statistics as salient. Predictions from the model corroborate the relationship between bottom-up auditory attention and statistical inference, and argues for a potential role of predictive coding as mechanism for saliency detection in acoustic scenes.

show abstract

A temporal saliency map for modeling auditory attention

Cited by 28 publications

References 13 publications

Modelling auditory attention

Modelling auditory attention

What Can Computational Models Learn From Human Selective Attention? A Review From an Audiovisual Unimodal and Crossmodal Perspective

Investigating bottom-up auditory attention

Contact Info

Product

Resources

About