Detecting conversational groups in images and sequences: A robust game-theoretic approach

Vascon, Sebastiano; Mequanint, Eyasu Zemene; Cristani, Marco; Hung, Hayley; Pelillo, Marcello; Murino, Vittorio

doi:10.1016/j.cviu.2015.09.012

Cited by 60 publications

(46 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dominant set framework is a pairwise clustering approach [41], based on the notion of a dominant set, which can be seen as an edge-weighted generalization of a clique. The approach is a fast and efficient framework for pairwise clustering and has been used to solve multiple problems, such as data association in tracking [42] as well as group detection [43].…”

Section: The Dominant Set Frameworkmentioning

confidence: 99%

Large-Scale Image Geo-Localization Using Dominant Sets

Zemene

Tesfaye

Idrees

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

This paper presents a new approach for the challenging problem of geo-locating an image using image matching in a structured database of city-wide reference images with known GPS coordinates. We cast the geo-localization as a clustering problem on local image features. Akin to existing approaches on the problem, our framework builds on low-level features which allow partial matching between images. For each local feature in the query image, we find its approximate nearest neighbors in the reference set. Next, we cluster the features from reference images using Dominant Set clustering, which affords several advantages over existing approaches. First, it permits variable number of nodes in the cluster which we use to dynamically select the number of nearest neighbors (typically coming from multiple reference images) for each query feature based on its discrimination value. Second, as we also quantify in our experiments, this approach is several orders of magnitude faster than existing approaches. Thus, we obtain multiple clusters (different local maximizers) and obtain a robust final solution to the problem using multiple weak solutions through constrained Dominant Set clustering on global image features, where we enforce the constraint that the query image must be included in the cluster. This second level of clustering also bypasses heuristic approaches to voting and selecting the reference image that matches to the query. We evaluate the proposed framework on an existing dataset of 102k street view images as well as a new larger dataset of 300k images, and show that it outperforms the state-of-the-art by 20% and 7%, respectively, on the two datasets.

show abstract

Section: The Dominant Set Frameworkmentioning

confidence: 99%

Large-Scale Image Geo-Localization Using Dominant Sets

Zemene

Tesfaye

Idrees

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Classically favoured modalities for F-formation detection has been static images and videos. Throughout the years, many different methodologies such as Hough voting [16], [17], graph clustering with dominant sets [18], [19] and multi-payoff evolutionary game theory [9] were used to detect F-formations from static images or video.…”

Section: Related Workmentioning

confidence: 99%

Detecting F-formations & Roles in Crowded Social Scenes with Wearables: Combining Proxemics & Dynamics using LSTMs

Rosatelli

Gedik

Hung

2019

2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)

Self Cite

View full text Add to dashboard Cite

In this paper, we investigate the use of proxemics and dynamics for automatically identifying conversing groups, or so-called F-formations. More formally we aim to automatically identify whether wearable sensor data coming from 2 people is indicative of F-formation membership. We also explore the problem of jointly detecting membership and more descriptive information about the pair relating to the role they take in the conversation (i.e. speaker or listener). We jointly model the concepts of proxemics and dynamics using binary proximity and acceleration obtained through a single wearable sensor per person. We test our approaches on the publicly available MatchNMingle dataset which was collected during real-life mingling events. We find out that fusion of these two modalities performs significantly better than them independently, providing an AUC of 0.975 when data from 30-second windows are used. Furthermore, our investigation into roles detection shows that each role pair requires a different time resolution for accurate detection.

show abstract

“…The F-Formation detection methods (Zhang and Hung, 2016;Cristani et al, 2011;Vascon et al, 2014Vascon et al, , 2016Hung and Kröse, 2011) use the people positions on the ground floor and orientations of part of people's body as initial features. Moreover, they have been split into two approaches according to the detection place: (1) still image, and (2) image sequences (i.e.…”

Section: Introductionmentioning

confidence: 99%

“…dominant set) problem (Zhang and Hung, 2016;Hung and Kröse, 2011). However, for F-Formation detection over video, according to our knowledge, only methods based on evolutionary game-theory clustering are reported (Vascon et al, 2014(Vascon et al, , 2016. In these methods, fusion and clustering steps are performed over an evolutionary environment, where the social representation is codified on an affinity matrix.…”

Section: Introductionmentioning

confidence: 99%

“…they use simplex method), the clustering step is O(pce) (Bulò and Pelillo, 2009), where c is the average number of iteration required for converging and e is the amount of social relations. Thus, the complexity of the method proposed in Vascon et al (2016) is O(kp 2 + 2 i + pce). For this reason, in this paper we propose a more efficient method for F-Formation detection.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detecting Free Standing Conversational Group in Video Using Fuzzy Relations

et al. 2019

View full text Add to dashboard Cite

In Computer Vision and Pattern Recognition, surveillance-video crowded scenes have been analysed according to their structure, where the detection of distinguishable people groups is an essential step. In this paper, we are interested in detecting F-Formations (i.e. free standing conversational groups) on video, which are formed by people social relations. We proposed a new method based on fuzzy relations, where a new social representation for computing relation between individuals, fusion for search consensus in multiple frame and clustering are introduced. Finally, our proposal was tested in a real-world dataset, improving the already reported scores from literature.

show abstract

Detecting conversational groups in images and sequences: A robust game-theoretic approach

Cited by 60 publications

References 45 publications

Large-Scale Image Geo-Localization Using Dominant Sets

Large-Scale Image Geo-Localization Using Dominant Sets

Detecting F-formations & Roles in Crowded Social Scenes with Wearables: Combining Proxemics & Dynamics using LSTMs

Detecting Free Standing Conversational Group in Video Using Fuzzy Relations

Contact Info

Product

Resources

About