Let me join you! Real-time F-formation recognition by a socially aware robot

Barua, Hrishav Bakul; Pramanick, Pradip; Sarkar, Chayan; Mg, Theint Haythi

doi:10.1109/ro-man47096.2020.9223469

Cited by 12 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the literature, many works have studied F-formation patterns either to investigate the different F-formation arrangements that emerged during an interaction between a robot and human [61][62][63] or to investigate the quality of interaction between the robot and humans [64][65][66]. However, only a handful of works have proposed approaches to recognise the spatial patterns [28,29], which are further used to interact with groups of people.…”

Section: Estimating F-formationsmentioning

confidence: 99%

“…The presented work performs better under two groups in the scene and only four patterns are studied. Recently, one more work has emerged, [28], there a machine-learning-based method is proposed to interact with the group of people. The approach considers poses of people in the group and uses the SVM classifier for predicting the F-formation.…”

Section: Estimating F-formationsmentioning

confidence: 99%

“…Recently, few works are proposed in regards to mobile robot using on-board sensors to interact with group. However, the number of groups in the scene is limited, i.e., one group [28] and two groups [29] with only two or three people per group, and limited patterns, i.e., four patterns, were studied. These research gaps suggest a need for robot-centric approaches, which detect groups and estimate F-formations to join the ongoing social group interactions in the scene.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Detecting Groups and Estimating F-Formations for Social Human–Robot Interactions

Pathi

Kiselev

Loutfi

2022

MTI

View full text Add to dashboard Cite

The ability of a robot to detect and join groups of people is of increasing importance in social contexts, and for the collaboration between teams of humans and robots. In this paper, we propose a framework, autonomous group interactions for robots (AGIR), that endows a robot with the ability to detect such groups while following the principles of F-formations. Using on-board sensors, this method accounts for a wide spectrum of different robot systems, ranging from autonomous service robots to telepresence robots. The presented framework detects individuals, estimates their position and orientation, detects groups, determines their F-formations, and is able to suggest a position for the robot to enter the social group. For evaluation, two simulation scenes were developed based on the standard real-world datasets. The 1st scene is built with 20 virtual agents (VAs) interacting in 7 different groups of varying sizes and 3 different formations. The 2nd scene is built with 36 VAs, positioned in 13 different groups of varying sizes and 6 different formations. A model of a Pepper robot is used in both simulated scenes in randomly generated different positions. The ability for the robot to estimate orientation, detect groups, and estimate F-formations at various locations is used to determine the validation of the approaches. The obtained results show a high accuracy within each of the simulated scenarios and demonstrates that the framework is able to work from an egocentric view with a robot in real time.

show abstract

Section: Estimating F-formationsmentioning

confidence: 99%

Section: Estimating F-formationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detecting Groups and Estimating F-Formations for Social Human–Robot Interactions

Pathi

Kiselev

Loutfi

2022

MTI

View full text Add to dashboard Cite

show abstract

“…Experimental HRI work has validated the idea that spatial formations typical of human-human conversations naturally emerge in human-robot interactions (Hüttenrauch et al, 2006;Kuzuoka et al, 2010;Karreman et al, 2015;Vázquez et al, 2015aVázquez et al, , 2017. In turn, this research led to work on recognizing F-Formations in robotics, such as methods geared toward improving robot navigation (Rios-Martinez et al, 2011), generating multimodal nonverbal robot behavior (Vázquez et al, 2017), helping recognize the beginning and ending of human-robot interactions (Gaschler et al, 2012), joining groups (Barua et al, 2020), and other approaches for service robots (Hedayati et al, 2019;Swofford et al, 2020). Oftentimes, prior work on F-Formation detection in robotics builds on mathematical models of human F-Formations from the computer vision community, for example, (Cristani et al, 2011;Setti et al, 2013;Setti et al, 2015;Vascon et al, 2014).…”

Section: Related Workmentioning

confidence: 99%

“…Although it is common to model conversational spatial behavior with discriminative models of group formations ( Truong and Ngo, 2017 ; Vázquez et al, 2017 ; Hedayati et al, 2019 ; Barua et al, 2020 ; Swofford et al, 2020 ), we approach the problem of predicting a pose for a robot in a group conversation with generative models. These models can directly output poses for the robot based on the social context of the interaction and spatial constraints imposed by the environment, for example, due to small objects such as tables or bigger structures such as walls.…”

Section: Introductionmentioning

confidence: 99%

Pose Generation for Social Robots in Conversational Group Formations

et al. 2022

View full text Add to dashboard Cite

We study two approaches for predicting an appropriate pose for a robot to take part in group formations typical of social human conversations subject to the physical layout of the surrounding environment. One method is model-based and explicitly encodes key geometric aspects of conversational formations. The other method is data-driven. It implicitly models key properties of spatial arrangements using graph neural networks and an adversarial training regimen. We evaluate the proposed approaches through quantitative metrics designed for this problem domain and via a human experiment. Our results suggest that the proposed methods are effective at reasoning about the environment layout and conversational group formations. They can also be used repeatedly to simulate conversational spatial arrangements despite being designed to output a single pose at a time. However, the methods showed different strengths. For example, the geometric approach was more successful at avoiding poses generated in nonfree areas of the environment, but the data-driven method was better at capturing the variability of conversational spatial formations. We discuss ways to address open challenges for the pose generation problem and other interesting avenues for future work.

show abstract

Relationship existence recognition-based social group detection in urban public spaces

Qing

Guo

et al. 2023

Neurocomputing

View full text Add to dashboard Cite

Let me join you! Real-time F-formation recognition by a socially aware robot

Cited by 12 publications

References 19 publications

Detecting Groups and Estimating F-Formations for Social Human–Robot Interactions

Detecting Groups and Estimating F-Formations for Social Human–Robot Interactions

Pose Generation for Social Robots in Conversational Group Formations

Relationship existence recognition-based social group detection in urban public spaces

Contact Info

Product

Resources

About