Investigating Graph-based Features for Speech Emotion Recognition

Pentari, Anastasia; Kafentzis, George P.; Tsiknakis, Manolis

doi:10.1109/bhi56158.2022.9926795

Cited by 6 publications

(9 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, our speech analysis relies on the graph-based theory, including structural and statistical information of a time series. Thus, we firstly introduce the extraction of the structural graph-based speech representation through the Visibility Graph (VG) theory [13]. After that, we extend our description to the statistical information computational approach.…”

Section: Methodsmentioning

confidence: 99%

“…The VG theory has been shown to be an appropriate tool for investigating and further determining the structural interrelations among the samples of a time series, which consists of positive defined elements [13]. Inspired by [12], we have applied VG theory to the SER problem in our previous work [13], which provided evidence regarding the effectiveness of VG theory in the analysis of speech signals. However, as discussed in [12], the VG has two disadvantages: first, it does not consider the effect of uneven sampling; second, it cannot capture the time series changes below a zero baseline.…”

Section: Structural Graph-based Speech Informationmentioning

confidence: 99%

“…Hence, graphs can be based on the statistical information of the signals or even their structural one, through a variety of approaches including the Pearson's correlation and the visibility graph theory, respectively. Lately, the graphbased theory was introduced to SER field, by exploiting the structural information and it was proved really promising [13]. Thus, in this study we aim to extend this previous work and provide a novel pipeline to analyze appropriately the speech signals.…”

Section: Introductionmentioning

confidence: 97%

“…The main approach of constructing these matrices is based on taking specific mathematical relations among segments of speech signals (of fixed as well as varying length). In this work we aim to extend our previous work proposed in [13]. That approach was the first evaluation of whether graph-based theory could be proved effective towards the SER problem.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Graph-Based Representations of Speech Signals: \\A Novel Approach for Emotion Recognition

Pentari

Kafentzis

Tsiknakis

2023

Preprint

View full text Add to dashboard Cite

Section: Methodsmentioning

confidence: 99%

Section: Structural Graph-based Speech Informationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 97%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Graph-Based Representations of Speech Signals: \\A Novel Approach for Emotion Recognition

Pentari

Kafentzis

Tsiknakis

2023

Preprint

View full text Add to dashboard Cite

“…At present, the application of graph neural networks in the field of speech technology still has some limitations [25], but some scholars have verified the advantages of graph convolution in the field of speech technology and the possibility of being widely used through research, such as conversational speech recognition [26], sentence-level [27] / conversation-level speech emotion recognition [28], speech enhancement [29], and Q &A rewriting [30]. The methods of graph construction can be divided into sample point-based, frame-based, speech channel-based, and historical dialogue-based approaches, as shown in Fig.…”

Section: Ser Based On Gnnsmentioning

confidence: 99%

Speech emotion recognition based on Graph-LSTM neural network

Li,

Wang,

Yang

et al. 2023

J AUDIO SPEECH MUSIC PROC.

View full text Add to dashboard Cite

Currently, Graph Neural Networks have been extended to the field of speech signal processing. It is the more compact and flexible way to represent speech sequences by graphs. However, the structures of the relationships in recent studies are tend to be relatively uncomplicated. Moreover, the graph convolution module exhibits limitations that impede its adaptability to intricate application scenarios. In this study, we establish the speech-graph using feature similarity and introduce a novel architecture for graph neural network that leverages an LSTM aggregator and weighted pooling. The unweighted accuracy of 65.39% and the weighted accuracy of 71.83% are obtained on the IEMOCAP dataset, achieving the performance comparable to or better than existing graph baselines. This method can improve the interpretability of the model to some extent, and identify speech emotion features effectively.

show abstract

Speech emotion recognition via graph-based representations

Pentari,

Kafentzis,

Tsiknakis

2024

Sci Rep

View full text Add to dashboard Cite

Speech emotion recognition (SER) has gained an increased interest during the last decades as part of enriched affective computing. As a consequence, a variety of engineering approaches have been developed addressing the challenge of the SER problem, exploiting different features, learning algorithms, and datasets. In this paper, we propose the application of the graph theory for classifying emotionally-colored speech signals. Graph theory provides tools for extracting statistical as well as structural information from any time series. We propose to use the mentioned information as a novel feature set. Furthermore, we suggest setting a unique feature-based identity for each emotion belonging to each speaker. The emotion classification is performed by a Random Forest classifier in a Leave-One-Speaker-Out Cross Validation (LOSO-CV) scheme. The proposed method is compared with two state-of-the-art approaches involving well known hand-crafted features as well as deep learning architectures operating on mel-spectrograms. Experimental results on three datasets, EMODB (German, acted) and AESDD (Greek, acted), and DEMoS (Italian, in-the-wild), reveal that our proposed method outperforms the comparative methods in these datasets. Specifically, we observe an average UAR increase of almost $$18\%$$ 18 % , $$8\%$$ 8 % and $$13\%$$ 13 % , respectively.

show abstract

Investigating Graph-based Features for Speech Emotion Recognition

Cited by 6 publications

References 32 publications

Graph-Based Representations of Speech Signals: \\A Novel Approach for Emotion Recognition

Graph-Based Representations of Speech Signals: \\A Novel Approach for Emotion Recognition

Speech emotion recognition based on Graph-LSTM neural network

Speech emotion recognition via graph-based representations

Contact Info

Product

Resources

About