Automatic Role Recognition in Multiparty Recordings: Using Social Affiliation Networks for Feature Extraction

Abstract

Abstract-Automatic analysis of social interactions attracts increasing attention in the multimedia community. This paper considers one of the most important aspects of the problem, namely the roles played by individuals interacting in different settings. In particular, this work proposes an automatic approach for the recognition of roles in both production environment contexts (e.g., news and talk-shows) and spontaneous situations (e.g., meetings). The experiments are performed over roughly 90 hours of material (one of the largest databases used for role recognition in the literature) and show that the recognition effectiveness depends on how much the roles influence the behavior of people. Furthermore, this work proposes the first approach for modeling mutual dependences between roles and assesses its effect on role recognition performance.

Human behavior analysis in video surveillance: A Social Signal Processing perspective

“…Such class includes five major components [11]: prosody, that can provide social signals like competence; linguistic vocalization, that can communicate hesitation; non linguistic vocalization, that can provide strong emotional states or tight social bonds, silence, that can express hesitation, and turn taking patterns: this last component is the most investigated in this category, since it appears the most reliable when the goal is to recognize people personality [68], predict the outcome of negotiations [69], recognize the roles interaction participants play [70], or modeling the type of interactions (e.g., a conflict).…”

Linking speaking and looking behavior patterns with group composition, perception, and performance

“…Personality of individuals in groups was estimated in [14]. Formal roles were automatically estimated in [16]. Dominant behavior was inferred in [7] and emergent leadership was studied in [17].…”

Automatic role recognition based on conversational and prosodic behaviour

“…In a few cases, the two approaches have been combined and some works propose movement based features (fidgeting) as well, resulting into multimodal approaches based on both audio and video analysis. Turn-taking has been used in [12,11], where temporal proximity of speakers is used to build social networks and extract features fed to Bayesian classifiers based on discrete distributions. Temporal proximity, and duration of interventions, are used in [3,10,6,5] as well, where they are combined with the ditribution of words in speech transcriptions.…”