Detecting F-formations &amp; Roles in Crowded Social Scenes with Wearables: Combining Proxemics &amp; Dynamics using LSTMs

Rosatelli, Alessio; Gedik, Ekin; Hung, Hayley

doi:10.1109/aciiw.2019.8925179

Cited by 8 publications

(9 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This includes advances in integrating active learning approaches in order to better personalize the prediction models for specific users based on their feedback. Lastly, this proposed architecture should be integrated with work on interpreting the meaning of social signals such as body language in social interactions [Rosatelli et al, 2019]. This would allow the agent to take into account the dynamics of a social situation as it unfolds, allowing it to integrate social situation understanding based on social relations with observed social signals.…”

Section: Discussionmentioning

confidence: 99%

“…Kola et al [2019] propose an approach to model arbitrary social situations through a two-level ontology distinguishing situation cues and social relationship features (social background model). Rosatelli et al [2019] propose an approach where data from wearable sensors is processed with deep learning techniques to assess information such as roles in social interactions.…”

Section: Level 1: Social Situation Perceptionmentioning

confidence: 99%

See 1 more Smart Citation

Who’s That? - Social Situation Awareness for Behaviour Support Agents

Kola

Jonker

Riemsdijk

2020

Engineering Multi-Agent Systems

View full text Add to dashboard Cite

Behaviour support agents need to be aware of the social environment of the user in order to be able to provide comprehensive support. However, this is a feature that is currently lacking in existing systems. To tackle it, first of all we explore literature from social sciences in order to find which elements of the social environment need to be represented. We structure this knowledge as a two-level ontology that models social situations. We formalize the elements that are needed to model social situations, which consist of different types of meetings between two people. We conduct an experiment to evaluate the lower level of the ontology using feedback from the subjects, and to test whether we can use the data to reason about the priority of different situations. Subjects found our proposed features of social relationships to be understandable and representative. Furthermore, we show these features can be combined in a decision tree to predict the priority of social situations.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Level 1: Social Situation Perceptionmentioning

confidence: 99%

Who’s That? - Social Situation Awareness for Behaviour Support Agents

Kola

Jonker

Riemsdijk

2020

Engineering Multi-Agent Systems

View full text Add to dashboard Cite

show abstract

“…ConfLab enables more robust models to be developed to conceptualize and detect social involvement. The use of the Chalcedony badges mentioned in the MatchNMingle dataset show more promising results using their radio-based proximity sensor and acceleration data [27]. However, they are still far away from performing sufficiently for more downstream tasks due to the relatively low sample frequency (20Hz) and annotation frequency (1Hz) [16].…”

Section: Related Workmentioning

confidence: 99%

ConfLab: A Rich Multimodal Multisensor Dataset of Free-Standing Social Interactions in the Wild

Raman¹,

Vargas-Quiros²,

Tan³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We describe an instantiation of a new concept for multimodal multisensor data collection of real life in-the-wild free standing social interactions in the form of a Conference Living Lab (ConfLab). ConfLab contains high fidelity data of 49 people during a real-life professional networking event capturing a diverse mix of status, acquaintanceship, and networking motivations at an international conference. Recording such a dataset is challenging due to the delicate trade-off between participant privacy and fidelity of the data, and the technical and logistic challenges involved. We improve upon prior datasets in the fidelity of most of our modalities: 8-camera overhead setup, personal wearable sensors recording body motion (9-axis IMU), Bluetooth-based proximity, and low-frequency audio. Additionally, we use a state-of-the-art hardware synchronization solution and timeefficient continuous technique for annotating body keypoints and actions at high frequencies. We argue that our improvements are essential for a deeper study of interaction dynamics at finer time scales. Our research tasks showcase some of the open challenges related to in-the-wild privacy-preserving social data analysis: keypoints detection from overhead camera views, skeleton based no-audio speaker detection, and F-formation detection. With the ConfLab dataset, we aim to bridge the gap between traditional computer vision tasks and in-the-wild ecologically valid socially-motivated tasks. * Equal contribution Preprint. Under review.

show abstract

“…In such a context, the most important cues are interpersonal distances and relative body orientations [72]. Furthermore, many works address the detection of Fformations [41,64,66,91,93,122,158,160,172,211], which are spatial arrangements that people spontaneously form during free-standing social interactions. In F-formations context, people communicating with one another tend to maintain a close proximity while having a shared focus of attention.…”

Section: Dyads and Groupsmentioning

confidence: 99%

Face-to-Face Co-Located Human-Human Social Interaction Analysis using Nonverbal Cues: A Survey

Beyan¹,

Vinciarelli²,

Bue³

2022

Preprint

View full text Add to dashboard Cite

This work presents a systematic review of recent efforts (since 2010) aimed at automatic analysis of nonverbal cues displayed in face-to-face co-located human-human social interactions. The main reason for focusing on nonverbal cues is that these are the physical, machine detectable traces of social and psychological phenomena. Therefore, detecting and understanding nonverbal cues means, at least to a certain extent, to detect and understand social and psychological phenomena. The covered topics are categorized into three as: a) modeling social traits, such as leadership, dominance, personality traits, b) social role recognition and social relations detection and c) interaction dynamics analysis in terms of group cohesion, empathy, rapport and so forth. We target the co-located interactions, in which the interactants are always humans. The survey covers a wide spectrum of settings and scenarios, including free-standing interactions, meetings, indoor and outdoor social exchanges, dyadic conversations, and crowd dynamics. For each of them, the survey considers the three main elements of nonverbal cues analysis, namely data, sensing approaches and computational methodologies. The goal is to highlight the main advances of the last decade, to point out existing limitations, and to outline future directions. CCS Concepts: • Human-centered computing → Collaborative and social computing; • Computing methodologies → Machine learning.

show abstract

Detecting F-formations & Roles in Crowded Social Scenes with Wearables: Combining Proxemics & Dynamics using LSTMs

Cited by 8 publications

References 30 publications

Who’s That? - Social Situation Awareness for Behaviour Support Agents

Who’s That? - Social Situation Awareness for Behaviour Support Agents

ConfLab: A Rich Multimodal Multisensor Dataset of Free-Standing Social Interactions in the Wild

Face-to-Face Co-Located Human-Human Social Interaction Analysis using Nonverbal Cues: A Survey

Contact Info

Product

Resources

About