H-Vectors: Utterance-Level Speaker Embedding Using a Hierarchical Attention Model

Shi, Yanpei; Huang, Qiang; Hain, Thomas

doi:10.1109/icassp40776.2020.9054448

Cited by 14 publications

(11 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…XI. For completeness we also mention h-vectors [152] which use a hierarchical attention mechanism to produce utterance-level embeddings, but has only been applied to speaker recognition tasks.…”

Section: Neural Network Embeddingsmentioning

confidence: 99%

Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview

Bell

Fainberg

Klejch

et al. 2021

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

Section: Neural Network Embeddingsmentioning

confidence: 99%

Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview

Bell

Fainberg

Klejch

et al. 2021

IEEE Open J. Signal Process.

View full text Add to dashboard Cite

“…In addition to feed-forward deep neural networks (DNNs) (Hinton et al, 2012), recurrent and convolutional models have also been applied to extract d-vectors at frame-level (Variani et al, 2014;Yella and Stolcke, 2015;Heigold et al, 2016;Cyrta et al, 2017;Wang et al, 2018b). To convert a variable length segment into a fixed-length vector using frame-level d-vectors, a temporal pooling function, such as the mean and standard deviation (Garcia-Romero et al, 2017;Diez et al, 2019;Wang et al, 2018c), attention mechanisms (Chowdhury et al, 2018;Zhu et al, 2018;Sun et al, 2019;Shi et al, 2020), and their combination (Okabe et al, 2018), have been used, which also enables joint training over entire segments.…”

Section: Related Workmentioning

confidence: 99%

Combination of Deep Speaker Embeddings for Diarisation

Sun,

Zhang,

Woodland

2020

Preprint

View full text Add to dashboard Cite

Combining deep speaker embedding (window-level d-vector) extraction systems using 2D self-attentive, gated additive and bilinear pooling methods. The best-performing structure for combination is obtained by stacking a 2D self-attentive and a bilinear pooling structures.• A complete single pass neural network-based diarisation pipeline is introduced, which includes neural voice activity detection, neural change point detection, a deep speaker embedding extraction system and spectral clustering.• Experiments on both AMI and NIST RT05 evaluation sets showed that our proposed methods can produce state-ofthe-art results for the very challenging multi-speaker (with 4-10 speakers) meeting diarisation task.• We use the AMI dataset based on the official speech recognition partition with the audios recorded by multiple distance microphones (MDM) since it is a more realistic setup for meeting transcription than many different setups used by previous AMI based studies. This is the first paper to use this setup to the best of the authors' knowledge. Although this makes our results not directly comparable to those from the previous papers, we think our system still shows superior performance, since the results shown in Table 11 are the lowest diarisation error rates with the same training data, and the realistic setup we used can increase the difficulty of the task.• This paper is made based on our previous conference paper (Sun et al., 2019). Compared to our previous paper -Almost all of the combination structures are newly proposed and not included in (Sun et al., 2019), apart from the first type of 2D self-attentive method defined in Eqn. (4).-Sun et al. ( 2019) only performed the experiments on the AMI data with manual segmentation, while this paper explores to use both manual and automatic segmentation. The neural diarisation pipeline is newly introduced in this paper, although the neural VAD structure was proposed in (Wang et al., 2016).

show abstract

“…Four strong baselines are chosen to compare with the proposed model: X-vectors [17], Attentive X-vector (Att-Xvector) [18,19,2,20], H-vector [6,21] and S-vector [22]. X-vectors is TDNN based model, which contains a TDNN based frame-level feature extractor, a statistics pooling operation and a segment-level feature extractor.…”

Section: Experiments Setupmentioning

confidence: 99%

T-vectors: Weakly Supervised Speaker Identification Using Hierarchical Transformer Model

Shi,

Chen,

Huang

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Identifying multiple speakers without knowing where a speaker's voice is in a recording is a challenging task. This paper proposes a hierarchical network with transformer encoders and memory mechanism to address this problem. The proposed model contains a framelevel encoder and segment-level encoder, both of them make use of the transformer encoder block. The multi-head attention mechanism in the transformer structure could better capture different speaker properties when the input utterance contains multiple speakers. The memory mechanism used in the frame-level encoders can build a recurrent connection that better capture long-term speaker features. The experiments are conducted on artificial datasets based on the Switchboard Cellular part1 (SWBC) and Voxceleb1 datasets. In different data construction scenarios (Concat and Overlap), the proposed model shows better performance comparaing with four strong baselines, reaching 13.3% and 10.5% relative improvement compared with H-vectors and S-vectors. The use of memory mechanism could reach 10.6% and 7.7% relative improvement compared with not using memory mechanism.

show abstract

H-Vectors: Utterance-Level Speaker Embedding Using a Hierarchical Attention Model

Cited by 14 publications

References 18 publications

Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview

Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview

Combination of Deep Speaker Embeddings for Diarisation

T-vectors: Weakly Supervised Speaker Identification Using Hierarchical Transformer Model

Contact Info

Product

Resources

About