Dynamic Behavior Analysis via Structured Rank Minimization

Abstract

Human behavior and affect is inherently a dynamic phenomenon involving temporal evolution of patterns manifested through a multiplicity of non-verbal behavioral cues including facial expressions, body postures and gestures, and vocal outbursts. A natural assumption for human behavior modeling is that a continuous-time characterization of behavior is the output of a linear time-invariant system when behavioral cues act as the input (e.g., continuous rather than discrete annotations of dimensional affect). Here we study the learning of such dynamical system under real-world conditions, namely in the presence of noisy behavioral cues descriptors and possibly unreliable annotations by employing structured rank minimization. To this end, a novel structured rank minimization method and its scalable variant are proposed. The generalizability of the proposed framework is demonstrated by conducting experiments on 3 distinct dynamic behavior analysis tasks, namely (i) conflict intensity prediction, (ii) prediction of valence and arousal, and (iii)

Recovering Joint and Individual Components in Facial Data

“…However, ADMM has been applied successfully in non-linear optimization problems in practice [14], [25], [26], [27], [28]. In addition, the thorough experimental evaluation of the proposed methods presented in Section 7, indicates that the obtained solutions are good for the data upon which RJIVE was tested.…”

Online Attention for Interpretable Conflict Estimation in Political Debates

“…Concretely, statistical model of verbal and acoustic features have been applied for disagreement detection [3][4] [5], while in [6] [7] the task is addressed by employing a sequential discriminative model. Kim et al [8] [9] employ audio features for conflict detection while methods for estimation of continuous-valued conflict intensity have been proposed in [10] [11] [8]. However, the aforementioned methods ignore or oversimplify the temporal dimension which is of utmost importance to the problems of conflict and (dis)agreement estimation.…”

Behavior prediction in-the-wild

“…Such systems, including affective multimodal interfaces, interactive multi-party games, and online services, would facilitate market research analysis, personalized e-commerce, and recruitment as well as enable patient-centric healthcare technologies such as emote monitoring of conditions like pain, anxiety and depression, to mention but a few examples. A fundamental pre-requisite for the development of interfaces like the above mentioned is the deployment of end-to-end machine learning frameworks capable of detecting, tracking, modeling, recognizing and predicting naturalistic -and, consequently, highly ambiguous -human behaviors [1]. Urged by this ever-growing necessity, this paper focuses on temporal dynamics-based behavior prediction in-the-wild, that is in naturalistic, unconstrained conditions.…”

“…The modeling assumption behind the proposed framework is that naturalistic affect and behavior captured in audiovisual episodes are smoothly-varying dynamic phenomena and thus the hidden temporal dynamics can be modeled as a generative auto-regressive process [1]. In particular, continuous-time real-valued characterizations of behavior (annotations) are postulated to be outputs of a low-complexity (i.e., low-order) time-invariant Linear Dynamical System (LDS) when descriptors of behavioral cues (features) act as inputs.…”

“…In particular, continuous-time real-valued characterizations of behavior (annotations) are postulated to be outputs of a low-complexity (i.e., low-order) time-invariant Linear Dynamical System (LDS) when descriptors of behavioral cues (features) act as inputs. First, the memory of the latent dynamic process in question, or, equivalently, the order of the underlying LTI system, is learned directly from the noisy training observations of inputs (features) and outputs (annotations) by applying the convex instance of the (Hankel)-structured matrix rank minimization model proposed in [1]. Next, a LDS is learned for each displayed behavior by solving a system of linear equations.…”