Social activity recognition based on probabilistic merging of skeleton features with proximity priors from RGB-D data

Abstract

Abstract-Social activity based on body motion is a key feature for non-verbal and physical behavior defined as function for communicative signal and social interaction between individuals. Social activity recognition is important to study human-human communication and also human-robot interaction. Based on that, this research has threefold goals: (1) recognition of social behavior (e.g. human-human interaction) using a probabilistic approach that merges spatio-temporal features from individual bodies and social features from the relationship between two individuals; (2) learn priors based on physical proximity between individuals during an interaction using proxemics theory to feed a probabilistic ensemble of activity classifiers; and (3) provide a public dataset with RGB-D data of social daily activities including risk situations useful to test approaches for assisted living, since this type of dataset is still missing. Results show that using the proposed approach designed to merge features with different semantics and proximity priors improves the classification performance in terms of precision, recall and accuracy when compared with other approaches that employ alternative strategies.

Automatic detection of human interactions from RGB-D data for social activity classification

“…work [1]. The description of the possible settings of a social interaction using body language has been deeply investigated in social science.…”

“…Like the individual activity recognition, also social activity recognition is capturing the attention of many researchers. Our previous work [1] presented an extension of DBMM for multiple mixtures, which is used to develop a system for the recognition of social activities by merging the models for two separate individuals and their social characteristics. These approaches were able to identify human activities with a good level of performance.…”

“…In [19], the authors proposed video clips of realistic individual activities and sub-activities including information about the used objects. In [1] we introduced a dataset for recognition of social activities. In [10], a dataset containing many video clips of 60 action classes, from 3 different views, including individual and social activities is presented.…”

“…A new dataset (UoL-3D Social Interaction Dataset) of social interaction between two subjects has been developed to evaluate the performance of our approach and to be made publicly available to the research community 1 . This dataset consists of RGB, Depth and tracked skeletons of the participants (i.e.…”

“…joints' 3D coordinates and quaternions), collected with a Kinect 2 sensor. It differs from the dataset presented in [1] for its realistic scenario and setting of the performed activities. Indeed, the activities are not performed as truncated (or repeated) clips, but as a continuous stream of activities, performed over several minutes.…”