Toward Explainable Affective Computing: A Review

Abstract

Affective computing has an unprecedented potential to change the way humans interact with technology. While the last decades have witnessed vast progress in the field, multimodal affective computing systems are generally black box by design. As affective systems start to be deployed in real-world scenarios, such as education or healthcare, a shift of focus toward improved transparency and interpretability is needed. In this context, how do we explain the output of affective computing models? and how to do so without limiting predictive performance? In this article, we review affective computing work from an explainable AI (XAI) perspective, collecting and synthesizing relevant papers into three major XAI approaches: premodel (applied before training), in-model (applied during training), and postmodel (applied after training). We present and discuss the most fundamental challenges in the field, namely, how to relate explanations back to multimodal and time-dependent data, how to integrate context and inductive biases into explanations using mechanisms such as attention, generative modeling, or graph-based methods, and how to capture intramodal and cross-modal interactions in post hoc explanations. While explainable affective computing is still nascent, existing methods are promising, contributing not only toward improved transparency but, in many cases, surpassing state-of-the-art results. Based on these findings, we explore directions for future research and discuss the importance of data-driven XAI and explanation goals, and explainee needs definition, as well as causability or the extent to which a given method leads to human understanding.

Transforming Education: A Comprehensive Review of Generative Artificial Intelligence in Educational Settings through Bibliometric and Content Analysis

“…Cortinas-Lorenzo and Lacey (2023) [26] Addresses the need for transparency and interpretability in affective computing systems, highlighting the challenges associated with multimodal data, context integration, and interaction capturing.…”

Machine learning analysis of thermophysical and thermohydraulic properties in ethylene glycol- and glycerol-based SiO2 nanofluids

“…The use of deep learning, and neural networks, in particular, provides a sophisticated method for locating intricate patterns within comprehensive datasets. The model is trained and tested on a split dataset, and statistics-based performance metrics are employed to evaluate the model's performance 22 , 23 . With the help of the latest methods in machine learning, like ensemble approaches the researchers can get a better understanding of the complex physics involved in heat transfer in SiO 2 nanofluids as a result of the insights they receive on the significance of several factors 24 – 26 .…”

Unlocking renewable energy potential: Harnessing machine learning and intelligent algorithms

“…A summary of recent ML based WtE studies These tactics are meant to provide an understanding of the model's decisions and guarantee that the projections are precise and understandable. It is somewhat difficult to strike the ideal balance between interpretability and model complexity(Cortiñas-Lorenzo and Lacey, 2024;Qin et al, 2023).…”