Multimodal Speaker Identification Using Canonical Correlation Analysis

Sargm, M. Emre; Erzin, Engin; Yemez, Y.; Tekalp, A. Murat

doi:10.1109/icassp.2006.1660095

Cited by 23 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [23] the authors apply classical linear transformations for dimensionality reduction (such as principal component analysis -PCA, or linear discriminant analysis-LDA) on feature vectors resulting from the concatenation of audio and visual speech features. CCA is used in [39] where projected audio and visual speech features are used as input for client-dependent HMM models.…”

Section: Identity Verificationmentioning

confidence: 99%

Audiovisual Speech Synchrony Measure: Application to Biometrics

Bredin

Chollet

2007

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

Speech is a means of communication which is intrinsically bimodal: the audio signal originates from the dynamics of the articulators. This paper reviews recent works in the field of audiovisual speech, and more specifically techniques developed to measure the level of correspondence between audio and visual speech. It overviews the most common audio and visual speech front-end processing, transformations performed on audio, visual, or joint audiovisual feature spaces, and the actual measure of correspondence between audio and visual speech. Finally, the use of synchrony measure for biometric identity verification based on talking faces is experimented on the BANCA database.

show abstract

Section: Identity Verificationmentioning

confidence: 99%

Audiovisual Speech Synchrony Measure: Application to Biometrics

Bredin

Chollet

2007

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

show abstract

“…Furthermore, semi-supervised [Hou et al 2010], sparse unsupervised [Han et al 2012] and spectral embedding [Feng et al 2017] based multi-view learning have also been introduced in CCA learning. The above reviewed CCA and variants have found wide applications in image processing [Ji et al 2015] ], multimedia analysis [Sargin et al 2006] [Sargin et al 2007] [Tae-Kyun andRoberto 2009], information retrieval [Vinokourov et al 2002], text classification [Rupnik and Grobelnik 2008], biometric recognition [Zhai et al 2012] [Haghighat et al 2016] [Xie et al 2016], human mobility modeling [Zhang et al 2017], etc.…”

Section: Introductionmentioning

confidence: 99%

Moment-Guided Discriminative Manifold Correlation Learning on Ordinal Data

Qiu

Zhang

Cao

et al. 2020

ACM Trans. Intell. Syst. Technol.

View full text Add to dashboard Cite

Canonical correlation analysis (CCA) is a typical and useful learning paradigm in big data analysis for capturing correlation across multiple views of the same objects. When dealing with data with additional ordinal information, traditional CCA suffers from poor performance due to ignoring the ordinal relationships within the data. Such data is becoming increasingly common, as either temporal or sequential information is often associated with the data collection process. To incorporate the ordinal information into the objective function of CCA, the so-called ordinal discriminative CCA has been presented in the literature. Although ordinal discriminative CCA can yield better ordinal regression results, its performance deteriorates when data is corrupted with noise and outliers, as it tends to smear the order information contained in class centers. To address this issue, in this article we construct a robust manifold-preserved ordinal discriminative correlation regression (rmODCR). The robustness is achieved by replacing the traditional ( l 2 -norm) class centers with l p -norm centers, where p is efficiently estimated according to the moments of the data distributions, as well as by incorporating the manifold distribution information of the data in the objective optimization. In addition, we further extend the robust manifold-preserved ordinal discriminative correlation regression to deep convolutional architectures. Extensive experimental evaluations have demonstrated the superiority of the proposed methods.

show abstract

“…Canonical correlation analysis (CCA) between these vectors is used to derive this transformation model. The CCA has been used previously for analysis of correlation among different features [22] and for fusion of multi-modal features in SV [23]. Additionally, it has been used for co-whitening for short and long duration utterances in an i-vector system [24].…”

Section: Introductionmentioning

confidence: 99%

Generative X-Vectors for Text-Independent Speaker Verification

Das

Yılmaz

et al. 2018

2018 IEEE Spoken Language Technology Workshop (SLT)

View full text Add to dashboard Cite

Speaker verification (SV) systems using deep neural network embeddings, so-called the x-vector systems, are becoming popular due to its good performance superior to the i-vector systems. The fusion of these systems provides improved performance benefiting both from the discriminatively trained x-vectors and generative i-vectors capturing distinct speaker characteristics. In this paper, we propose a novel method to include the complementary information of i-vector and x-vector, that is called generative x-vector. The generative x-vector utilizes a transformation model learned from the i-vector and x-vector representations of the background data. Canonical correlation analysis is applied to derive this transformation model, which is later used to transform the standard x-vectors of the enrollment and test segments to the corresponding generative x-vectors. The SV experiments performed on the NIST SRE 2010 dataset demonstrate that the system using generative x-vectors provides considerably better performance than the baseline i-vector and x-vector systems. Furthermore, the generative x-vectors outperform the fusion of i-vector and x-vector systems for long-duration utterances, while yielding comparable results for short-duration utterances.

show abstract

Multimodal Speaker Identification Using Canonical Correlation Analysis

Cited by 23 publications

References 12 publications

Audiovisual Speech Synchrony Measure: Application to Biometrics

Audiovisual Speech Synchrony Measure: Application to Biometrics

Moment-Guided Discriminative Manifold Correlation Learning on Ordinal Data

Generative X-Vectors for Text-Independent Speaker Verification

Contact Info

Product

Resources

About