Deep neural networks for Multi-Room Voice Activity Detection: Advancements and comparative evaluation

Vesperini, Fabio; Vecchiotti, Paolo; Principi, Emanuele; Squartini, Stefano; Piazza, Francesco

doi:10.1109/ijcnn.2016.7727633

Cited by 17 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In all cases, HMM-based Viterbi decoding and "w-sum" decision fusion are used, where the combined log-likelihoods result from microphone-specific GMMs (left) or a GMM trained on a single microphone (right) In comparison, our proposed algorithm exhibits SAD errors of 5.7% and 4.7%, when operating over entire segments ("seg") or sliding windows ("win"), respectively. The latter represents a 19% relative SAD error reduction over the DNN of [30] and 10% over the 3D-CNN of [32], proving better than segment-based operation in the challenging and noisy DIRHA-sim-evalita data (as also observed in Table 6 for DIRHA-sim). These comparisons highlight the competitiveness of our two-stage system and the suitability of the five room discriminant features of its second stage.…”

Section: Comparison To Deep Learning Approachessupporting

confidence: 54%

“…Specifically, in [29], a DNN is employed taking as input 176-dimensional vectors composed of a variety of features, such as MFCCs, RASTA-PLPs, envelope variance, and pitch. Similar features (but 187-dimensional) and DNNs are again considered in [30], as well as alternative classifiers, including a 2D-CNN. The latter is extended to a multi-channel 3D-CNN system in [31], where log-Mel filterbank energies (40-dimensional) are employed as features, temporal context is exploited by concatenating adjacent time frames, and the resulting 2D single-microphone feature matrices are stacked across channels.…”

Section: Related Workmentioning

confidence: 99%

“…As overviewed in Section 2, a number of works on room-localized SAD have appeared recently, proposing single-stage algorithms based on deep learning methods [29][30][31][32]. In this section, a performance comparison to our developed system is provided.…”

Section: Comparison To Deep Learning Approachesmentioning

confidence: 99%

“…Comparative results between the best deep learning results of [29][30][31][32] and our proposed algorithm are presented in Table 8. In particular, the best system of for the speech-inside vs. speech-outside classification task with ground-truth segmentation on the DIRHA-sim test set, using various numbers of microphones (en, energy-based; coh, coherence-based; srp, SRP-based; ts, spectrogram texture smoothness; ev, envelope variance; all, intra-room fusion (9)) Inter-room fusion (10) and room-specific SVMs are used [29,30], employing a DNN over 187-dimensional features of various types that are extracted from the best microphone per room, yields a SAD error of 5.8%. Further, the 3D-CNN system of [31], operating on 40-dimensional log-Mel filterbank energies after temporal splicing and combining information from the three best microphones per room, exhibits a 7.0% SAD error.…”

Section: Comparison To Deep Learning Approachesmentioning

confidence: 99%

“…Several efforts have been reported recently on roomlocalized SAD in multi-room environments [25][26][27][28][29][30][31][32] including our own work [33,34]. As further overviewed in Section 2, these approaches vary in the kind of features, classifiers, and number of microphones used per room.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Room-localized speech activity detection in multi-microphone smart homes

Giannoulis

Potamianos

Maragos

2019

J AUDIO SPEECH MUSIC PROC.

View full text Add to dashboard Cite

Voice-enabled interaction systems in domestic environments have attracted significant interest recently, being the focus of smart home research projects and commercial voice assistant home devices. Within the multi-module pipelines of such systems, speech activity detection (SAD) constitutes a crucial component, providing input to their activation and speech recognition subsystems. In typical multi-room domestic environments, SAD may also convey spatial intelligence to the interaction, in addition to its traditional temporal segmentation output, by assigning speech activity at the room level. Such room-localized SAD can, for example, disambiguate user command referents, allow localized system feedback, and enable parallel voice interaction sessions by multiple subjects in different rooms. In this paper, we investigate a room-localized SAD system for smart homes equipped with multiple microphones distributed in multiple rooms, significantly extending our earlier work. The system employs a two-stage algorithm, incorporating a set of hand-crafted features specially designed to discriminate room-inside vs. room-outside speech at its second stage, refining SAD hypotheses obtained at its first stage by traditional statistical modeling and acoustic front-end processing. Both algorithmic stages exploit multi-microphone information, combining it at the signal, feature, or decision level. The proposed approach is extensively evaluated on both simulated and real data recorded in a multi-room, multi-microphone smart home, significantly outperforming alternative baselines. Further, it remains robust to reduced microphone setups, while also comparing favorably to deep learning-based alternatives.

show abstract

Section: Comparison To Deep Learning Approachessupporting

confidence: 54%

Section: Related Workmentioning

confidence: 99%

Section: Comparison To Deep Learning Approachesmentioning

confidence: 99%

Section: Comparison To Deep Learning Approachesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Room-localized speech activity detection in multi-microphone smart homes

Giannoulis

Potamianos

Maragos

2019

J AUDIO SPEECH MUSIC PROC.

View full text Add to dashboard Cite

show abstract

Convolutional Neural Networks with 3-D Kernels for Voice Activity Detection in a Multiroom Environment

Vecchiotti

Vesperini

Principi

et al. 2017

Multidisciplinary Approaches to Neural Computing

Self Cite

View full text Add to dashboard Cite

This paper focuses on employing Convolutional Neural Networks (CNN) with 3-D kernels for Voice Activity Detectors in multi-room domestic scenarios (mVAD). This technology is compared with the Multi Layer Perceptron (MLP) and interesting advancements are observed with respect to previous works of the authors. In order to approximate real-life scenarios, the DIRHA dataset is exploited. It has been recorded in a home environment by means of several microphones arranged in various rooms. Our study is composed by a multi-stage analysis focusing on the selection of the network size and the input microphones in relation with their number and position. Results are evaluated in terms of Speech Activity Detection error rate (SAD). The CNN-mVAD outperforms the other method with a significant solidity in terms of performance statistics, achieving in the best overall case a SAD equal to 7.0%.

show abstract

An Interpersonal Dynamics Analysis Procedure With Accurate Voice Activity Detection Using Low-Cost Recording Sensors

Lim,

Kang,

Choi

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Voice data is of special interest to organizational behavior researchers since it can be easily collected and provides a wealth of information for understanding interpersonal dynamics. Due to these benefits, voice activity detection (VAD) using wearable sensor badges has received considerable attention as one of the more objective and effective data-driven analysis methods rather than self-report methods for analyzing interpersonal dynamics. Moreover, with the VAD results, several prior works extracted conversational features, such as speaking times, turn-taking, and overlapped speaking, for attaining highlevel organizational insights. However, there is still room for improvement in the accuracy of the VAD models, and there is a lack of research to develop reliable conversational feature extracting algorithms in the previous works. Most of these prior studies relied on sociometric badges, which are costly electronic devices, including voice recording machines and redundant sensors. In this paper, we propose an interpersonal dynamics analysis procedure based on low-cost commercial recording sensors, consisting of data-driven VAD modeling and conversational feature extracting. To accurately identify voice presence, the VAD modeling incorporates three steps: signal preprocessing, data-derived generation, and VAD with machine learning-based classification and smoothing technique. We conducted experiments, from collecting datasets comprising the human verbal interaction of 15 three-person groups by commercial recording sensors to implementing our procedure on the datasets. Our results show that the proposed procedure excels in voice activity detection, achieving superior accuracy compared to prior studies. This remarkable accuracy subsequently ensures greater reliability in our conversational feature extraction. Thus, the proposed procedure can encourage organizational behavior researchers to acquire objective information about interpersonal dynamics and efficiently obtain high-level organizational insights with cost-effectiveness.

show abstract

Deep neural networks for Multi-Room Voice Activity Detection: Advancements and comparative evaluation

Cited by 17 publications

References 24 publications

Room-localized speech activity detection in multi-microphone smart homes

Room-localized speech activity detection in multi-microphone smart homes

Convolutional Neural Networks with 3-D Kernels for Voice Activity Detection in a Multiroom Environment

An Interpersonal Dynamics Analysis Procedure With Accurate Voice Activity Detection Using Low-Cost Recording Sensors

Contact Info

Product

Resources

About