Applying Multitask Deep Learning to Emotion Recognition in Speech

Abstract

Purpose of research. Emotions play one of the key roles in the regulation of human behaviour. Solving the problem of automatic recognition of emotions makes it possible to increase the effectiveness of operation of a whole range of digital systems such as security systems, human-machine interfaces, e-commerce systems, etc. At the same time, the low efficiency of modern approaches to recognizing emotions in speech can be noted. This work studies automatic recognition of emotions in speech applying machine learning methods.Methods. The article describes and tests an approach to automatic recognition of emotions in speech based on multitask learning of deep convolution neural networks of AlexNet and VGG architectures using automatic selection of the weight coefficients for each task when calculating the final loss value during learning. All the models were trained on a sample of the IEMOCAP dataset with four emotional categories of ‘anger’, ‘happiness’, ‘neutral emotion’, ‘sadness’. The log-mel spectrograms of statements processed by a specialized algorithm are used as input data.Results. The considered models were tested on the basis of numerical metrics: the share of correctly recognized instances, accuracy, completeness, f-measure. For all of the above metrics, an improvement in the quality of emotion recognition by the proposed model was obtained in comparison with the two basic single-task models as well as with known solutions. This result is achieved through the use of automatic weighting of the values of the loss functions from individual tasks when forming the final value of the error in the learning process.Conclusion. The resulting improvement in the quality of emotion recognition in comparison with the known solutions confirms the feasibility of applying multitask learning to increase the accuracy of emotion recognition models. The developed approach makes it possible to achieve a uniform and simultaneous reduction of errors of individual tasks, and is used in the field of emotions recognition in speech for the first time.

Modern automatic recognition technologies for visual communication tools

“…), жестами и движениями тела. В [143] отмечается важность внешне наблюдаемой специфической двигательной активности для задач автоматического распознавания эмоций.…”

Increased Performance of Transformers Language Models in Information Question and Response Systems

“…Часто в этих чатах пользователи общаются непосредственно с людьми, которые были наняты для помощи пользователям. Однако сейчас можно заметить, как множество компаний работают над внедрением искусственного интеллекта во взаимодей-ствие с клиентами [1], а также над замещением реальных людей в службе поддержки на различные модели машинного обучения [1]. Поскольку данные модели должны уметь находить нужную для пользователей информацию в различных базах знаний, то одной из важных задач в обработке естественного языка является задача нахождения ответа на вопрос в тексте.…”