A Study on Online Source Extraction in the Presence of Changing Speaker Positions

Heitkaemper, Jens; Fehér, Thomas; Freitag, Michael; Haeb‐Umbach, Reinhold

doi:10.1007/978-3-030-31372-2_17

Cited by 4 publications

(6 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In most current neural TSE works, the target speaker's location is assumed to be fixed. Although the methods should be easily extended to a dynamic case, investigations of such settings remain relatively rare [24].…”

Section: E Discussionmentioning

confidence: 99%

“…Furthermore, the flexibility of NNs to integrate different modalities combined with the high modeling capability of face recognition or lip-reading systems offered new possibilities for speaker-open visual clue-based TSE [7], [8]. More recently, neural approaches have also been introduced for spatial-clue-based TSE [3], [24].…”

Section: Historical Contextmentioning

confidence: 99%

“…More recent IVE models use location for initialization [12]. Finally, several works have shown that NNs informed by location can also achieve promising performance [3], [24]. Spatial clues are inherently applicable only when a recording from multiple microphones is available.…”

Section: A Type Of Cluementioning

confidence: 99%

“…All approaches with spatial clues require using a microphone array to capture the direction information of the sources in the mixture [2], [3], [16], [24], [36]. Some TSE approaches that exploit audio or visual clues also assume multi-channel recordings, such as the extensions of ICA/IVA approaches [5], [12].…”

Section: B Number Of Microphonesmentioning

confidence: 99%

“…Recent single-channel neural TSE systems have achieved remarkable performance. Interestingly, such approaches can also be easily extended to multi-channel processing by augmenting the input with spatial features [3] or combining the processing with beamforming [24], [30], as discussed in Section IV-C. For example, using a beamformer usually extracts a higher quality signal due to employing a spatial linear filter to perform extraction, which can benefit ASR applications [10].…”

Section: B Number Of Microphonesmentioning

confidence: 99%

See 4 more Smart Citations

Neural Target Speech Extraction: An Overview

Žmolíková¹,

Delcroix²,

Ochiai³

et al. 2023

Preprint

View full text Add to dashboard Cite

Humans can listen to a target speaker even in challenging acoustic conditions that have noise, reverberation, and interfering speakers. This phenomenon is known as the cocktail-party effect. For decades, researchers have focused on approaching the listening ability of humans. One critical issue is handling interfering speakers because the target and non-target speech signals share similar characteristics, complicating their discrimination. Target speech/speaker extraction (TSE) isolates the speech signal of a target speaker from a mixture of several speakers with or without noises and reverberations using clues that identify the speaker in the mixture. Such clues might be a spatial clue indicating the direction of the target speaker, a video of the speaker's lips, or a pre-recorded enrollment utterance from which their voice characteristics can be derived. TSE is an emerging field of research that has received increased attention in recent years because it offers a practical approach to the cocktailparty problem and involves such aspects of signal processing as audio, visual, array processing, and deep learning. This paper focuses on recent neural-based approaches and presents an in-depth overview of TSE. We guide readers through the different major approaches, emphasizing the similarities among frameworks and discussing potential future directions.

show abstract

Section: E Discussionmentioning

confidence: 99%