CoMet: Modeling Group Cohesion for Socially Compliant Robot Navigation in Crowded Scenes

Sathyamoorthy, Adarsh Jagan; Patel, Utsav; Paul, Moumita; Kumar, Nithish K Sanjeev; Savle, Yash; Manocha, Dinesh

doi:10.1109/lra.2021.3135560

Cited by 15 publications

(14 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Much research has been conducted on identifying group features and group discovery [1]- [4]. With video as input, [2] uses YOLOv2 for object detection, converts the detected objects into feature vectors, obtains a feature matrix, and uses it as input for clustering, grouping nearby people at the spatial level.…”

Section: A Human Group Learningmentioning

confidence: 99%

“…It allows the robot to anticipate potential collisions or other interactions and adjust its trajectory accordingly. However, these algorithms have limitations, such as needing to carry out expensive online computations or having reduced accuracy under certain conditions [1]. For example, when the crowd density is high, computing multiple features online takes too long and negatively impacts the performance of the navigation algorithm.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SG-LSTM: Social Group LSTM for Robot Navigation Through Dense Crowds

Bhaskara¹,

Chiu²,

Bera³

2023

Preprint

View full text Add to dashboard Cite

With the increasing availability and affordability of personal robots, they will no longer be confined to large corporate warehouses or factories but will instead be expected to operate in less controlled environments alongside larger groups of people. In addition to ensuring safety and efficiency, it is crucial to minimize any negative psychological impact robots may have on humans and follow unwritten social norms in these situations. Our research aims to develop a model that can predict the movements of pedestrians and perceptuallysocial groups in crowded environments. We introduce a new Social Group Long Short-term Memory (SG-LSTM) model that models human groups and interactions in dense environments using a socially-aware LSTM to produce more accurate trajectory predictions. Our approach enables navigation algorithms to calculate collision-free paths faster and more accurately in crowded environments. Additionally, we also release a large video dataset with labeled pedestrian groups for the broader social navigation community. We show comparisons with different metrics on different datasets (ETH, Hotel, MOT15) and different prediction approaches (LIN, LSTM, O-LSTM, S-LSTM) as well as runtime performance.

show abstract

Section: A Human Group Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

SG-LSTM: Social Group LSTM for Robot Navigation Through Dense Crowds

Bhaskara¹,

Chiu²,

Bera³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…For the navigation of social robots, parameters such as the trajectory, position, or speed of the movements of people or the robot itself were considered, but they did not take into account the emotions of multiple people [ 12 , 56 , 57 , 58 ]. There are studies that consider the influence of a robot within a group of people [ 13 , 16 , 59 ], but the detection of group emotions was not carried out and even less the detection of the emotion of an environment. There are very few studies proposing methods for group emotion estimation.…”

Section: Related Workmentioning

confidence: 99%

“…Developing systems with this perspective makes the robot able to adapt to social groups of humans [ 12 ]. The detection of groups of people improves the navigation of a social robot in indoor and outdoor environments, and the detection of group emotions allows the robot to improve HRI, exhibiting acceptable social behaviour [ 13 , 14 , 15 , 16 ], as well as associating the group emotion with the scene in which the group is participating. Nevertheless, most existing studies related to detecting group emotions are based on third-person cameras [ 17 , 18 , 19 , 20 , 21 ], but their complexity makes them unsuitable for social robots with egocentric vision due to their sensory capacity.…”

Section: Introductionmentioning

confidence: 99%

Group Emotion Detection Based on Social Robot Perception

Quiroz

Patiño

Díaz-Amado

et al. 2022

Sensors

View full text Add to dashboard Cite

Social robotics is an emerging area that is becoming present in social spaces, by introducing autonomous social robots. Social robots offer services, perform tasks, and interact with people in such social environments, demanding more efficient and complex Human–Robot Interaction (HRI) designs. A strategy to improve HRI is to provide robots with the capacity of detecting the emotions of the people around them to plan a trajectory, modify their behaviour, and generate an appropriate interaction with people based on the analysed information. However, in social environments in which it is common to find a group of persons, new approaches are needed in order to make robots able to recognise groups of people and the emotion of the groups, which can be also associated with a scene in which the group is participating. Some existing studies are focused on detecting group cohesion and the recognition of group emotions; nevertheless, these works do not focus on performing the recognition tasks from a robocentric perspective, considering the sensory capacity of robots. In this context, a system to recognise scenes in terms of groups of people, to then detect global (prevailing) emotions in a scene, is presented. The approach proposed to visualise and recognise emotions in typical HRI is based on the face size of people recognised by the robot during its navigation (face sizes decrease when the robot moves away from a group of people). On each frame of the video stream of the visual sensor, individual emotions are recognised based on the Visual Geometry Group (VGG) neural network pre-trained to recognise faces (VGGFace); then, to detect the emotion of the frame, individual emotions are aggregated with a fusion method, and consequently, to detect global (prevalent) emotion in the scene (group of people), the emotions of its constituent frames are also aggregated. Additionally, this work proposes a strategy to create datasets with images/videos in order to validate the estimation of emotions in scenes and personal emotions. Both datasets are generated in a simulated environment based on the Robot Operating System (ROS) from videos captured by robots through their sensory capabilities. Tests are performed in two simulated environments in ROS/Gazebo: a museum and a cafeteria. Results show that the accuracy in the detection of individual emotions is 99.79% and the detection of group emotion (scene emotion) in each frame is 90.84% and 89.78% in the cafeteria and the museum scenarios, respectively.

show abstract

“…According to [4] as many as 215,660 people received warnings for violating health protocols at tourist attractions during May 2021. Therefore, in maximizing the application of health protocols [5] has created a COVID-Robot technology that can detect distance in a crowd and body temperature. However, the output of the COVID-Robot has not been able to directly warn of violations of health protocols.…”

Section: Introductionmentioning

confidence: 99%

Crowd Detection System Using Blimp Drones as an Effort to Mitigate the Spread of Covid-19 Based on Internet of Things

Mashoedah

Dhewa²,

Roviyan³

et al. 2021

ELINVO

View full text Add to dashboard Cite

The application of health protocols is a regulation that is applied to prevent the spread of Covid-19. Public awareness of the implementation of health protocols is still lacking. This study aims to determine the performance of the detection system using the Blimp Drone as an effort to mitigate the spread of Covid-19 based on the Internet of Things. The method used in system development consists of literature review, needs analysis, design, manufacture, and testing. This system uses the Blimp Drone as a vehicle to carry out flight missions. Raspberry pi camera as a component for distance detection in crowds and mask detection, and Flir Lepton 2.5 as a component for temperature detection. In addition, thealso used raspberry pi 4 is as a component for image processing of the distance detection system for crowd, mask, and temperature detection. The results of the tests carried out are: (1) the results of the analysis of the lifting power of the Blimp Drone using pure helium and a diameter of 0.91m capable of lifting a load of 0.45kg (2) the system can detect the distance of each person in the crowd (3) the system can detect everyone who using a mask and not wearing a mask (4) the system can detect the temperature of each person using image processing (5) an audio warning sounds when a person who is not wearing a mask is detected and the distance between 2 people in a crowd of less than 1 meter is detected (6) the advantages of this tool ie can work autonomous in flight missions to detect the distance the crowd, masks, and temperature, as well as integrated with system the Internet of Things so that data the results capturing can be accessed in real time for further action by the authorities.

show abstract

CoMet: Modeling Group Cohesion for Socially Compliant Robot Navigation in Crowded Scenes

Cited by 15 publications

References 33 publications

SG-LSTM: Social Group LSTM for Robot Navigation Through Dense Crowds

SG-LSTM: Social Group LSTM for Robot Navigation Through Dense Crowds

Group Emotion Detection Based on Social Robot Perception

Crowd Detection System Using Blimp Drones as an Effort to Mitigate the Spread of Covid-19 Based on Internet of Things

Contact Info

Product

Resources

About