Informed vs. Blind beamforming in AD-HOC Acoustic Sensor Networks for Meeting Transcription

Abstract

We propose a diarization system, that estimates "who spoke when" based on spatial information, to be used as a front-end of a meeting transcription system running on the signals gathered from an acoustic sensor network (ASN). Although the spatial distribution of the microphones is advantageous, exploiting the spatial diversity for diarization and signal enhancement is challenging, because the microphones' positions are typically unknown, and the recorded signals are initially unsynchronized in general. Here, we approach these issues by first blindly synchronizing the signals and then estimating time differences of arrival (TDOAs). The TDOA information is exploited to estimate the speakers' activity, even in the presence of multiple speakers being simultaneously active. This speaker activity information serves as a guide for a spatial mixture model, on which basis the individual speaker's signals are extracted via beamforming. Finally, the extracted signals are forwarded to a speech recognizer. Additionally, a novel initialization scheme for spatial mixture models based on the TDOA estimates is proposed. Experiments conducted on real recordings from the LibriWASN data set have shown that our proposed system is advantageous compared to a system using a spatial mixture model, which does not make use of external diarization information.

Efficient Joint Optimization of Sampling Rate Offsets Using Entire Multichannel Signal

“…where 1/(1 + ϵ m ) is approximated by 1 − ϵ m under the assumption of ϵ m ≪ 1 [13]. In (2), the effect of STO is modeled by a time-invariant shift of r 0 ∆ m samples.…”

“…WASNs can utilize common devices, including smartphones and tablets, to construct a microphone array without a dedicated multichannel analog-to-digital converter (ADC). Motivated by this flexibility and handiness, various audio technologies have been developed on the basis of WASNs such as speech enhancement [3]- [6], sound source separation [7]- [10], meeting transcription [11]- [13], and acoustic scene analysis [14], [15].…”

“…Blind SRO estimation, which uses only observed signals to estimate SROs, is desirable in various scenarios [11]- [13]. It is a challenging task owing to the lack of time stamps, calibration signals, and prior information on microphone and source positions.…”