Your attention, please! Determining saliency of competing audio stimuli in natural scenarios

Tordini, Francesco; Bregman, Albert S.; Ankolekar, Anupriya; Sandholm, Thomas; Cooperstock, Jeremy R.

doi:10.1121/1.4800201

Cited by 3 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tordini et al . [47,48] approached the problem from the opposite direction: while Kim et al . used no prior knowledge of acoustical features to guide their feature estimation, Tordini et al .…”

Section: Models Of Auditory Attentionmentioning

confidence: 99%

“…These studies had subjects choose the more salient of two presented scenes [33,35,38], and used a variety of natural environmental sounds as the salient events. Later models adapted more sophisticated paradigms where the background had a predictable structure, and the task was the detection of the salient event, which had a deviation from the predictable pattern [40,47], such as a violin note popping out of a stream of piano notes. While these efforts provide a structured way of investigating the precise characteristics of salience perception, their artificial structure limits their account of salience in realistic settings.…”

Section: Validation Of Auditory Attention Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Modelling auditory attention

Kaya¹,

Elhilali²

2017

Phil. Trans. R. Soc. B

109

101

View full text Add to dashboard Cite

Sounds in everyday life seldom appear in isolation. Both humans and machines are constantly flooded with a cacophony of sounds that need to be sorted through and scoured for relevant information—a phenomenon referred to as the ‘cocktail party problem’. A key component in parsing acoustic scenes is the role of attention, which mediates perception and behaviour by focusing both sensory and cognitive resources on pertinent information in the stimulus space. The current article provides a review of modelling studies of auditory attention. The review highlights how the term attention refers to a multitude of behavioural and cognitive processes that can shape sensory processing. Attention can be modulated by ‘bottom-up’ sensory-driven factors, as well as ‘top-down’ task-specific goals, expectations and learned schemas. Essentially, it acts as a selection process or processes that focus both sensory and cognitive resources on the most relevant events in the soundscape; with relevance being dictated by the stimulus itself (e.g. a loud explosion) or by a task at hand (e.g. listen to announcements in a busy airport). Recent computational models of auditory attention provide key insights into its role in facilitating perception in cluttered auditory scenes.This article is part of the themed issue ‘Auditory and visual scene analysis’.

show abstract

Section: Models Of Auditory Attentionmentioning

confidence: 99%

Section: Validation Of Auditory Attention Modelsmentioning

confidence: 99%

Modelling auditory attention

Kaya¹,

Elhilali²

2017

Phil. Trans. R. Soc. B

109

101

View full text Add to dashboard Cite

show abstract

“…A smaller body of work focuses on modeling bottomup auditory attention, or the qualities of a sound signal that make it likely to be attended to. Works in this domain construct representations of audio signals and evaluate their ability to reflect human judgment of saliency [32,56,58,61,65,97,98], distinguish between changes in perceptual qualities of sound [55], or perform on a downstream speech recognition task [54]. In this work, we focus on top-down, or selective auditory attention.…”

Section: Related Workmentioning

confidence: 99%

Egocentric Auditory Attention Localization in Conversations

Ryan¹,

Jiang²,

Shukla³

et al. 2023

Preprint

View full text Add to dashboard Cite

In a noisy conversation environment such as a dinner party, people often exhibit selective auditory attention, or the ability to focus on a particular speaker while tuning out others. Recognizing who somebody is listening to in a conversation is essential for developing technologies that can understand social behavior and devices that can augment human hearing by amplifying particular sound sources. The computer vision and audio research communities have made great strides towards recognizing sound sources and speakers in scenes. In this work, we take a step further by focusing on the problem of localizing auditory attention targets in egocentric video, or detecting who in a camera wearer's field of view they are listening to. To tackle the new and challenging Selective Auditory Attention Localization problem, we propose an end-to-end deep learning approach that uses egocentric video and multichannel audio to predict the heatmap of the camera wearer's auditory attention. Our approach leverages spatiotemporal audiovisual features and holistic reasoning about the scene to make predictions, and outperforms a set of baselines on a challenging multi-speaker conversation dataset. Project page: https://fkryan.github.io/saal

show abstract

“…The resulting perception of the auditory scene is composed of mental representations of the acoustic environment that are, in part, weighted according to their saliency (Kayser et al, 2005). Once segregation has occurred and layers of musical material have formed, layer selection is a competitive process in which the most salient between two concurrent layers is the one that is most likely to attract attention (Tordini et al, 2013).…”

Section: Attention and Auditory Saliencymentioning

confidence: 99%

“…Or in a more musical setting, a bright and strident trumpet may attract attention through its inherence saliency, but if it is embedded in an orchestral texture of other high brass instruments, it would stand out less than against the string sections. Saliency is difficult to define, as it exists between the realms of topdown (schema-driven) and bottom-up (stimulus-driven) attentional mechanisms (Tordini et al, 2013). To date, there are very few studies investigating the behavioral relevance of auditory saliency using naturalistic stimuli.…”

Section: Attention and Auditory Saliencymentioning

confidence: 99%

Instrument Timbre Combinations Influence The Relative Prominence of Perceptual Layers in Orchestral Music

Fischer,

McAdams

2023

Preprint

View full text Add to dashboard Cite

The stratification of layers of differing prominence (foreground/background) is a common technique in orchestration. Musicians heard 23 excerpts containing foreground and background layers as previously determined by music analysts. A given layer comprised either a single auditory stream of one or more blended instruments or a harmonic or rhythmic background. Two-layer excerpts had either the same, overlapping, or different instrument families (timbre class). First, musicians rated the perceived degree of segregation of musical materials in two-layer and single-stream excerpts. Second, they heard each of the two layers in isolation and then together and rated the relative prominence of the layers. Heterogeneous instrument combinations yielded the greatest difference in relative prominence, followed by overlapping and then homogeneous combinations. Acoustic and score-based descriptors were extracted to quantify their relative contribution to perceptual stratification. Timbre class and between-layer differences in timbre and dynamics played a role, providing evidence of how timbral differences enhance relative prominence in orchestral music. Perceptual segregation was positively but very weakly related to relative prominence, supporting findings that although segregation is necessary to form layers, this mechanism is separable from that which places the streams into the same representational space to allow for the assessment of their relative prominence.

show abstract

Your attention, please! Determining saliency of competing audio stimuli in natural scenarios

Cited by 3 publications

References 0 publications

Modelling auditory attention

Modelling auditory attention

Egocentric Auditory Attention Localization in Conversations

Instrument Timbre Combinations Influence The Relative Prominence of Perceptual Layers in Orchestral Music

Contact Info

Product

Resources

About