Phonetically-Aware Embeddings, Wide Residual Networks with Time-Delay Neural Networks and Self Attention Models for the 2018 NIST Speaker Recognition Evaluation

Viñals, Ignacio; Ribas, Dayana; Mingote, Victoria; Llombart, Jorge; Gimeno, Pablo; Miguel, Antonio; Giménez, Alfonso Ortega; Lleida, Eduardo

doi:10.21437/interspeech.2019-2417

Cited by 6 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, this architecture needs positional information [7] for the self attention layers to provide a good performance. Instead of using temporal positional information as many language modelling applications, we use the output of a phonetic classifier bottleneck [10,14]. The architecture of the phonetic classifier is an evolution of [10].…”

Section: Meanmentioning

confidence: 99%

“…Instead of using temporal positional information as many language modelling applications, we use the output of a phonetic classifier bottleneck [10,14]. The architecture of the phonetic classifier is an evolution of [10]. In this system, we use a modification of efficient net [17] to operate with 1D group convolutions as backbone.…”

Section: Meanmentioning

confidence: 99%

“…The success achieved with this approach in SV may be due to this kind of mechanism allows models to learn the frame-level representations, which are more precise to represent the speaker characteristics. Recently, multi-head attention proposed in [7] for Transformer architecture is a powerful attention mechanism which allows using several single attention mechanisms to extract diverse information over different parts of the network [10,11]. Furthermore, to improve the model capacity while computational efficiency is kept similar, memory layers [12] have been introduced in combination with the multi-head attention mechanism with success for language modelling tasks.…”

Section: Introductionmentioning

confidence: 99%

“…This memory layer is a productkey attention mechanism which allows storing the knowledge learned by the DNN during the training process. Moreover, we have added more phonetic information to the different architectures with the use of phonetic embeddings extracted from a phonetic classification network [10,14]. These phonetic embeddings are used as a complement to the feature extractor and can be used by the dot attention mechanism to locate phoneme similarities which can play the role of the positional embedding that is not used in our architecture.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Memory Layers with Multi-Head Attention Mechanisms for Text-Dependent Speaker Verification

Mingote

Miguel

Giménez

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

In this paper, we explore an approach based on memory layers and multi-head attention mechanisms to improve in an efficient way the performance of text-dependent speaker verification (SV) systems. The most extended SV systems based on Deep Neural Networks (DNN) extract the embedding of the utterance from the average pooling of the temporal dimension after processing. Unlike previous works, we can exploit the phonetic knowledge needed for text-dependent SV systems by combining the temporal attention of multiple parallel heads with the phonetic embeddings extracted from a phonetic classification network, which helps to guide to the attention mechanism with the role of the positional embedding. The addition of a memory layer to a text-dependent SV system was tested on the RSR2015-part II and DeepMine-part I databases, where, in both cases outperformed the baseline result and the reference system based on the same transformer network without the memory layer.

show abstract

Section: Meanmentioning

confidence: 99%

Section: Meanmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Memory Layers with Multi-Head Attention Mechanisms for Text-Dependent Speaker Verification

Mingote

Miguel

Giménez

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…From each original utterance we obtain phoneme labels, one per input feature vector. These phoneme labels in this experiment were obtained by automatic means, that is, a DNN phoneme classifier [35] consisting of a Wide Residual Network [36] with four blocks. Only 39 phoneme labels were considered, that is, each phoneme label includes all its associated coarticulation.…”

Section: Reduction Of the Mismatch In α: Phonetic Balancementioning

confidence: 99%

An Analysis of the Short Utterance Problem for Speaker Characterization

et al. 2019

Self Cite

View full text Add to dashboard Cite

Speaker characterization has always been conditioned by the length of the evaluated utterances. Despite performing well with large amounts of audio, significant degradations in performance are obtained when short utterances are considered. In this work we present an analysis of the short utterance problem providing an alternative point of view. From our perspective the performance in the evaluation of short utterances is highly influenced by the phonetic similarity between enrollment and test utterances. Both enrollment and test should contain similar phonemes to properly discriminate, being degraded otherwise. In this study we also interpret short utterances as incomplete long utterances where some acoustic units are either unbalanced or just missing. These missing units are responsible for the speaker representations to be unreliable. These unreliable representations are biased with respect to the reference counterparts, obtained from long utterances. These undesired shifts increase the intra-speaker variability, causing a significant loss of performance. According to our experiments, short utterances (3–60 s) can perform as accurate as if long utterances were involved by just reassuring the phonetic distributions. This analysis is determined by the current embedding extraction approach, based on the accumulation of local short-time information. Thus it is applicable to most of the state-of-the-art embeddings, including traditional i-vectors and Deep Neural Network (DNN) xvectors.

show abstract

Canonical EEG microstates transitions reflect switching among BOLD resting state networks and predict fMRI signal

et al. 2021

View full text Add to dashboard Cite

Objective. Electroencephalography microstates (EEG-ms), which reflect a large topographical representation of coherent electrophysiological brain activity, are widely adopted to study cognitive processes mechanisms and aberrant alterations in brain disorders. EEG-ms topographies are quasi-stable lasting between 60-120 milliseconds. Some evidence suggests that EEG-ms are the electrophysiological signature of resting-state networks (RSNs). However, the spatial and functional interpretation of EEG-ms and their association with functional MRI (fMRI) remains unclear. Approach. In a large cohort of healthy subjects (n = 52), we conducted several statistical and machine learning approaches analyses on the association among EEG-ms spatio-temporal dynamics and the blood-oxygenation-level dependent (BOLD) simultaneous EEG-fMRI data using statistical and machine learning approaches. Main results. Our results using a generalized linear model unraveled that EEG-ms transitions were largely and negatively associated with blood-oxygenation-level dependent (BOLD) signals in the somatomotor, visual, dorsal attention, and ventral attention fMRI networks with limited association within the default mode network. Additionally, a novel recurrent neural network (RNN) confirmed the association between EEG-ms transitioning and fMRI signal while revealing that EEG-ms dynamics can predict BOLD signals and vice versa. Significance. Results suggest that EEG-ms transitions may represent the deactivation of fMRI RSNs and provide evidence that both modalities can measure common aspects of undergoing brain neuronal activities. Moreover, our results may help to better understand the electrophysiological interpretation of EEG-ms and solve several contradicting findings in the literature.

show abstract

Phonetically-Aware Embeddings, Wide Residual Networks with Time-Delay Neural Networks and Self Attention Models for the 2018 NIST Speaker Recognition Evaluation

Cited by 6 publications

References 13 publications

Memory Layers with Multi-Head Attention Mechanisms for Text-Dependent Speaker Verification

Memory Layers with Multi-Head Attention Mechanisms for Text-Dependent Speaker Verification

An Analysis of the Short Utterance Problem for Speaker Characterization

Canonical EEG microstates transitions reflect switching among BOLD resting state networks and predict fMRI signal

Contact Info

Product

Resources

About