Using AI to predict service agent stress from emotion patterns in service interactions

Abstract

PurposeA vast body of literature has documented the negative consequences of stress on employee performance and well-being. These deleterious effects are particularly pronounced for service agents who need to constantly endure and manage customer emotions. The purpose of this paper is to introduce and describe a deep learning model to predict in real-time service agent stress from emotion patterns in voice-to-voice service interactions.Design/methodology/approachA deep learning model was developed to identify emotion patterns in call center interactions based on 363 recorded service interactions, subdivided in 27,889 manually expert-labeled three-second audio snippets. In a second step, the deep learning model was deployed in a call center for a period of one month to be further trained by the data collected from 40 service agents in another 4,672 service interactions.FindingsThe deep learning emotion classifier reached a balanced accuracy of 68% in predicting discrete emotions in service interactions. Integrating this model in a binary classification model, it was able to predict service agent stress with a balanced accuracy of 80%.Practical implicationsService managers can benefit from employing the deep learning model to continuously and unobtrusively monitor the stress level of their service agents with numerous practical applications, including real-time early warning systems for service agents, customized training and automatically linking stress to customer-related outcomes.Originality/valueThe present study is the first to document an artificial intelligence (AI)-based model that is able to identify emotions in natural (i.e. nonstaged) interactions. It is further a pioneer in developing a smart emotion-based stress measure for service agents. Finally, the study contributes to the literature on the role of emotions in service interactions and employee stress.

When do service employees smile? Response‐dependent emotion regulation in emotional labor

“…The most fundamental contribution is in showing the organizational viability of real‐time monitoring of affective displays of employees and customers. There are already initial mushrooms of such use in B2B support systems for service delivery where automated monitoring helps preempt cases of customer dissatisfaction and employee stress (e.g., http://loris.ai) as well as suggestions that automated analyses can facilitate managerial interventions in service delivery (Bromuri et al, 2021).…”

Does Customers’ Emotion toward Voice-based Service AI Cause Negative Reactions? Empirical Evidence from a Call Center

“…The second aspect is the study on the effect of AI on employees. AI applications can predict service agent stress from emotion's patterns in service interactions [26]. Employees and AI applications actively complement each other [27].…”

“…First, it extends the literature on customer emotion and service AI. Previous studies mainly focused on customers' emotions toward human agents (e.g., [26], [32], [57]), and how to use the AI to measure or analyze customer emotions [26,28]. However, the impact of customers' emotions on service AI remains deficient in the literature.…”

Intelligent Speech Recognition and Sentiment Analysis Considering LSTM Neural Network

“…The second step is to preprocess the speech signal, because the speech signal is analog continuous, so before the model recognition, the speech signal needs to be transformed into a discrete digital speech signal; After the continuous speech signal is converted to digital signal, the speech features are extracted. Finally, the speech recognition algorithm is selected, and the recognition model is established to obtain the speech ER rate [9,10]. The scheme flow of ER by speech signals is shown in Figure 1:…”