Smart Room: Participant and Speaker Localization and Identification

Multimodal Speaker Segmentation in Presence of Overlapped Speech Segments

“…As it is shown in [10] and [6], in the spherical coordinate system with the origin in the center of the small radius microphone array, source localization techniques based on the TDoA determine angular coordinates of the source much more precisely than its radial coordinate. Therefore, we model the distribution p(y MA t,i |x t,rt,i ) as a function of the angular distance (e.g.…”

“…[10,8]. In our previous work [6] we have presented algorithm that performs such tracking with high accuracy. In order to focus on aspects of microphone array processing and modality fusion, in this publication we track color markers (mini-paper hats) on the participant's heads through 4 ceiling cameras.…”

“…Recently proposed methods [13,3] suggest that the speaker segmentation based on microphone array estimation of the direction of arrival (DoA) [4,7] outperforms segmentation based on SID techniques [19]. However, the usual monitoring setups in meeting environments include small on-the-desk microphone arrays with limited spatial resolution [6,10] making it difficult to accurately disambiguate densely-spaced speakers based on DoA cues only. Furthermore, most methods rarely handle overlapped speech at the modeling level and fail to advantageously combine the microphone array and the SID observations.…”

“…Particularly, the system that we proposed in [6] can track unknown number of participants based on combination of the ceiling camera background subtraction and omnidirectional camera face tracking systems. In this work we have opted for a simpler solution, tracking of color markers, in order to focus on the advances in fusion and microphone array processing.…”

“…Our audio-visual smart-room system [6] enables tracking of locations and identities, and performs speaker segmentation, for a variable number of participants. It employs (Figure 1.…”