Customer Satisfaction Estimation in Contact Center Calls Based on a Hierarchical Multi-Task Model

Ando, Atsushi; Masumura, Ryo; Kamiyama, Hosana; Kobashikawa, Satoshi; Aono, Yushi; Toda, Tomoki

doi:10.1109/taslp.2020.2966857

Cited by 23 publications

(17 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Oliver [3] posited that expectation is the baseline or reference level for a customer to evaluate a product or service. Ando et al [5] demonstrated that the degree of customer satisfaction indicates the service quality of enterprises and their performance in customer engagement. Satisfaction is the cognitive assessment of expectation versus actual performance, and satisfaction is achieved when the level of perceived performance is higher than the initial level of expectation.…”

Section: A Satisfactionmentioning

confidence: 99%

See 1 more Smart Citation

Confirming Customer Satisfaction With Tones of Speech

Ko¹,

Hsu²,

Liu³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

As customer satisfaction explicitly leads to repurchase behavior, the level of customer satisfaction affects both sales performance and enterprise growth. Traditionally, measuring satisfaction requires customers spending extra time to fill out a post-purchase questionnaire survey. Recently, ASR (Automatic Speech Recognition) is utilized to extract spoken words from conversation to measure customer satisfaction. However, as oriental people tend to use vague words to express emotion, the approach has its limitation. To solve the problem, this study strived to complete following tasks: devising a process to collect customer voice expressing satisfaction and corresponding verifiable ground truth; a data bank of 150 customer voices speaking in Mandarin was collected; the features of the voice data were extracted with MFCC; as the size of data bank was limited, Auto Encoder was utilized to further reduce the features of voices; models based on Long Short-Term Memory (LSTM) Recurrent Neural Networks (RNNs) and Support Vector Machine (SVM) were constructed to predict satisfaction. With nested cross validation, the average accuracy of LSTM and SVM could reach 71.95% and 73.97%, respectively.

show abstract

Section: A Satisfactionmentioning

confidence: 99%

“…Satisfied customers are typically those who consider that their expectations are fulfilled. Therefore, evaluating customer satisfaction with each transaction is essential for enterprises to not only improve their service quality but also gain a clearer understanding of customer expectations [5].…”

Section: Introductionmentioning

confidence: 99%

Confirming Customer Satisfaction With Tones of Speech

Ko¹,

Hsu²,

Liu³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Therefore, speech emotion recognition (SER) is becoming a powerful technology to develop because of its enormous potential to achieve more natural human-computer and human-human interactions. SER is an essential component of many speechbased applications, such as call-center analysis [1]- [4], spoken dialog systems [5], [6], and education [7].…”

Section: Introductionmentioning

confidence: 99%

Speech Emotion Recognition Based on Self-Attention Weight Correction for Acoustic and Text Features

Santoso

Ishizuka²,

Hashimoto³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Speech emotion recognition (SER) is essential for understanding a speaker's intention. Recently, some groups have attempted to improve SER performance using a bidirectional long short-term memory (BLSTM) to extract features from speech sequences and a self-attention mechanism to focus on the important parts of the speech sequences. SER also benefits from combining the information in speech with text, which can be accomplished automatically using an automatic speech recognizer (ASR), further improving its performance. However, ASR performance deteriorates in the presence of emotion in speech. Although there is a method to improve ASR performance in the presence of emotional speech, it requires the fine-tuning of ASR, which has a high computational cost and leads to the loss of cues important for determining the presence of emotion in speech segments, which can be helpful in SER. To solve these problems, we propose a BLSTM-and-self-attention-based SER method using self-attention weight correction (SAWC) with confidence measures. This method is applied to acoustic and text feature extractors in SER to adjust the importance weights of speech segments and words with a high possibility of ASR error. Our proposed SAWC reduces the importance of words with speech recognition error in the text feature while emphasizing the importance of speech segments containing these words in acoustic features. Our experimental results on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) dataset reveal that our proposed method achieves a weighted average accuracy of 76.6%, outperforming other state-ofthe-art methods. Furthermore, we investigated the behavior of our proposed SAWC in each of the feature extractors.

show abstract

“…Thus, speech emotion recognition (SER) is an important technology for natural human-computer interaction. There are a lot of SER applications such as voice-of-customer analysis in contact center calls [1,2], driver state monitoring [3], and human-like responses in spoken dialog systems [4].…”

Section: Introductionmentioning

confidence: 99%

Speech emotion recognition based on listener-dependent emotion perception models

Ando

Mori

Kobashikawa

et al. 2021

SIP

Self Cite

View full text Add to dashboard Cite

This paper presents a novel speech emotion recognition scheme that leverages the individuality of emotion perception. Most conventional methods simply poll multiple listeners and directly model the majority decision as the perceived emotion. However, emotion perception varies with the listener, which forces the conventional methods with their single models to create complex mixtures of emotion perception criteria. In order to mitigate this problem, we propose a majority-voted emotion recognition framework that constructs listener-dependent (LD) emotion recognition models. The LD model can estimate not only listener-wise perceived emotion, but also majority decision by averaging the outputs of the multiple LD models. Three LD models, fine-tuning, auxiliary input, and sub-layer weighting, are introduced, all of which are inspired by successful domain-adaptation frameworks in various speech processing tasks. Experiments on two emotional speech datasets demonstrate that the proposed approach outperforms the conventional emotion recognition frameworks in not only majority-voted but also listener-wise perceived emotion recognition.

show abstract

Customer Satisfaction Estimation in Contact Center Calls Based on a Hierarchical Multi-Task Model

Cited by 23 publications

References 30 publications

Confirming Customer Satisfaction With Tones of Speech

Confirming Customer Satisfaction With Tones of Speech

Speech Emotion Recognition Based on Self-Attention Weight Correction for Acoustic and Text Features

Speech emotion recognition based on listener-dependent emotion perception models

Contact Info

Product

Resources

About