Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition

Abstract

Temporal context is key to the recognition of expressions of emotion. Existing methods, that rely on recurrent or selfattention models to enforce temporal consistency, work on the feature level, ignoring the task-specific temporal dependencies, and fail to model context uncertainty. To alleviate these issues, we build upon the framework of Neural Processes to propose a method for apparent emotion recognition with three key novel components: (a) probabilistic contextual representation with a global latent variable model; (b) temporal context modelling using task-specific predictions in addition to features; and (c) smart temporal context selection. We validate our approach on four databases, two for Valence and Arousal estimation (SEWA and AffWild2), and two for Action Unit intensity estimation (DISFA and BP4D). Results show a consistent improvement over a series of strong baselines as well as over state-ofthe-art methods.

Contextual Intelligent Decisions: Expert Moderation of Machine Outputs for Fair Assessment of Commercial Driving

“…Although not always easy to capture, some of the affective states brought out by experts during our workshops can be automatically detected using machine learning. Deep learning approaches applied to driver-facing footage, for instance, have shown promising results in automatically identifying some of the affective states, such as, distracted or attentive driving [43], [44], different types of human emotions [45], [46], and tired or energetic [47], [48]. Alternatively, calm or aggressive driving is accurately detected using telematics incident data [2], [3], while more complex affective states such as confidence and insecurity are still difficult to detect.…”