Automatic detection of human interactions from RGB-D data for social activity classification

Abstract

Abstract-We present a system for temporal detection of social interactions. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications, it is important to be able to move to more realistic data. For this reason, the proposed approach temporally detects intervals where individual or social activity is occurring. Recognition of human activities is a key feature for analysing the human behaviour. In particular, recognition of social activities is useful to trigger human-robot interactions or to detect situations of potential danger. Based on that, this research has three goals: (1) define a new set of descriptors, which are able to characterise human interactions; (2) develop a computational model to segment temporal intervals with social interaction or individual behaviour; (3) provide a public dataset with RGB-D data with continuous stream of individual activities and social interactions. Results show that the proposed approach attained relevant performance with temporal segmentation of social activities.

Indoor Bluetooth Low Energy Dataset for Localization, Tracking, Occupancy, and Social Interaction

“…The datasets of traces with implicit connections are Hope [49], composed of RFID tracking data collected from the seventh HOPE conference attendees carrying RFID badges, and a virtual mobility dataset composed of the traces of the virtual characters in the SecondLife virtual world [50]. A dataset providing fine-grained information, without using cameras [51,52], about the precise location of a socialization event in indoor environment is still missing.…”

Understanding of Human Behavior with a Robotic Agent Through Daily Activity Analysis

“…In a more recent work, Coppola et.al. [10] introduced a method for automatic detection of human interaction from RGB-D data that enabled social activities classification. In this work the authors defined a new set of descriptors, suitable to operate in realistic data, while developed a computational model to segment temporal intervals with social interaction or individual behavior and tested the method on their own publicly available dataset.…”

Social Activity Recognition on Continuous RGB-D Video Sequences

“…-Segmentation features: used to detect the temporal intervals of the social interactions (X Seg ) , based on the upper bodies of the two actors and originally proposed by [5]. These features are computed on two dimensions only (x and z of the Kinect 2 optical frame, see Fig.…”

“…Differently from a previous "3D Social Activity Dataset" by [6], the social activities in this new dataset appear in uninterrupted sequences, within the same video, alternating 2 or 3 social activities with individual ones such as read, phonecall, drink or sit. Furthermore, unlike the dataset introduced in [5], which was focused exclusively on the segmentation, the occurrence of all social activities is consistent in every video and the number of activities is higher, allowing to perform experiments for the performance evaluation of the classifier. The activities of this dataset, therefore, are not manually selected and cropped in short video clips, as in previous cases.…”

“…how ones movement affects the other) and the individual's movement information. Furthermore, in [5], a SVM-HMM model is used to segment the intervals of time in which social interactions occur. Since the performance of these two models has been only evaluated individually, their combined performance needs to be assessed to consider using them in robotic applications.…”