MultiMediate 2023: Engagement Level Detection using Audio and Video Features

Abstract

Participants in conversations may either actively engage in and enjoy the process, or become disinterested and allow their minds to wander. Measuring the engagement level of the participants can be quantified and used for analyzing numerous conversation scenarios such as business negotiations and remote learning, where the engagement level may indicate the success of negotiations or reflect the quality of the learning process. This paper presents a novel approach to engagement estimation using a versatile, resourceefficient, end-to-end training network. The network leverages body motion and audio characteristics, addressing challenges in data quality and model interpretability. Validated using the Noxi database and tested in the MultiMediate'23 competition, our approach achieved state-of-the-art performance, improving the baseline model's concordance coefficients correlation from 59% to 70% on the test set.