Exploiting the intermittency of speech for joint separation and diarization

Kounades-Bastian, Dionyssos; Girin, Laurent; Alameda-Pineda, Xavier; Horaud, Radu; Gannot, Sharon

doi:10.1109/waspaa.2017.8169991

Cited by 6 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are also recent researches to jointly perform speech separation and speaker diarization. Kounades-Bastian et al [165,166] proposed to incorporate a speech activity model into speech separation based on the spatial covariance model with non-negative matrix factorization. They derived the EM algorithm to estimate separated speech and speech activity of each speaker from the multi-channel overlapped speech.…”

Section: Joint Speech Separation and Diarizationmentioning

confidence: 99%

A Review of Speaker Diarization: Recent Advances with Deep Learning

Park¹,

Kanda²,

Dimitriadis³

et al. 2021

Preprint

View full text Add to dashboard Cite

Speaker diarization is a task to label audio or video recordings with classes corresponding to speaker identity, or in short, a task to identify "who spoke when". In the early years, speaker diarization algorithms were developed for speech recognition on multispeaker audio recordings to enable speaker adaptive processing, but also gained its own value as a stand-alone application over time to provide speaker-specific meta information for downstream tasks such as audio retrieval. More recently, with the rise of deep learning technology that has been a driving force to revolutionary changes in research and practices across speech application domains in the past decade, more rapid advancements have been made for speaker diarization. In this paper, we review not only the historical development of speaker diarization technology but also the recent advancements in neural speaker diarization approaches. We also discuss how speaker diarization systems have been integrated with speech recognition applications and how the recent surge of deep learning is leading the way of jointly modeling these two components to be complementary to each other. By considering such exciting technical trends, we believe that it is a valuable contribution to the community to provide a survey work by consolidating the recent developments with neural methods and thus facilitating further progress towards a more efficient speaker diarization.

show abstract

Section: Joint Speech Separation and Diarizationmentioning

confidence: 99%

A Review of Speaker Diarization: Recent Advances with Deep Learning

Park¹,

Kanda²,

Dimitriadis³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Very recently, joint processing of the two tasks have been proposed in [64] (for over-determined mixtures) and in [68,69]. The models in [68] and [69] combine a diarization state model (that encodes the combination of active sources within a given set of maximum size N ) with the multichannel LGM+NMF model of [107] and with the full-rank spatial covariance matrix model of [37], respectively. In contrast to [108,58], modelling the activity of all sources jointly using a diarization state enables to exploit the potential correlations on speaker activity.…”

Section: Varying Number Of (Active) Sourcesmentioning

confidence: 99%

Audio source separation into the wild

Girin

Gannot

2019

Multimodal Behavior Analysis in the Wild

Self Cite

View full text Add to dashboard Cite

This review chapter is dedicated to multichannel audio source separation in real-life environment. We explore some of the major achievements in the field and discuss some of the remaining challenges. We will explore several important practical scenarios, e.g. moving sources and/or microphones, varying number of sources and sensors, high reverberation levels, spatially diffuse sources, and synchronization problems. Several applications such as smart assistants, cellular phones, hearing aids and robots, will be discussed. Our perspectives on the future of the field will be given as concluding remarks of this chapter.

show abstract

“…Here, block-offline processing is allowed to utilize future data, while the block-online processing is not. In [17], joint separation and diarization is attempted using spatial mixture models. This, however, requires multichannel input and does not exploit spectral information for speaker re-identification.…”

Section: Introductionmentioning

confidence: 99%

All-neural Online Source Separation, Counting, and Diarization for Meeting Analysis

Neumann

Kinoshita

Delcroix

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Automatic meeting analysis comprises the tasks of speaker counting, speaker diarization, and the separation of overlapped speech, followed by automatic speech recognition. This all has to be carried out on arbitrarily long sessions and, ideally, in an online or block-online manner. While significant progress has been made on individual tasks, this paper presents for the first time an all-neural approach to simultaneous speaker counting, diarization and source separation. The NN-based estimator operates in a block-online fashion and tracks speakers even if they remain silent for a number of time blocks, thus learning a stable output order for the separated sources. The neural network is recurrent over time as well as over the number of sources. The simulation experiments show that state of the art separation performance is achieved, while at the same time delivering good diarization and source counting results. It even generalizes well to an unseen large number of blocks.

show abstract

Exploiting the intermittency of speech for joint separation and diarization

Cited by 6 publications

References 25 publications

A Review of Speaker Diarization: Recent Advances with Deep Learning

A Review of Speaker Diarization: Recent Advances with Deep Learning

Audio source separation into the wild

All-neural Online Source Separation, Counting, and Diarization for Meeting Analysis

Contact Info

Product

Resources

About