COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality

Zhou, Haifang; Kadav, Asim; Shamsian, Aviv; Geng, Shijie; Lai, Farley; Zhao, L.; Liu, Ting; Kapadia, Mubbasir; Graf, Hans Peter

doi:10.1007/978-3-031-19833-5_15

Cited by 20 publications

(11 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this research, a novel approach to group activity analysis is introduced, leveraging a fusion of the Yolov5 [12] and ViT[13] models to architect networks dedicated to individual and group activity analyses.The experimental outcomes showcase the efficacy of this innovative approach in real-world systems, offering a straightforward yet highly efficient solution within the realm of group activity analysis.Moving forward, future research endeavors will concentrate on enhancing the model's recognition accuracy while concurrently streamlining its architecture for lighter computational load.This endeavor aims to enable real-time deployment in practical scenarios, extending the applicability of this technology across diverse domains.Such advancements promise a broader spectrum of practical applications, fostering the widespread adoption of group activity analysis technology across multifarious industries.…”

Section: Discussionmentioning

confidence: 99%

“…Furthermore, some studies have amalgamated skeletal information with graph or semantic models for analyzing group behavior, as observed in works by Zhang et al [11] . and Zhou et al [12] . Overall, as the field of group activity analysis evolves, researchers have explored an array of methodologies and model structures to tackle diverse group activity recognition challenges.In the course of continuous development, only a handful of practical techniques have emerged, largely due to the complexity of proposed analytical methods that fail to align with practical application demands.…”

Section: Introductionmentioning

confidence: 92%

See 1 more Smart Citation

Exploration of group activity recognition methods and system implementation

Ye,

Feng,

Wang

et al. 2024

International Conference on Computer Graphics, Artificial Intelligence, and Data Processing (ICCAID 2023)

View full text Add to dashboard Cite

Group activity analysis holds diverse applications across various domains, encompassing daily behavior scrutiny, sports event investigation, and social behavior analysis. What sets the methodology proposed in this paper apart in group behavior analysis is its dual focus: not only does it scrutinize the behavioral trends exhibited by the collective but also delves into the nuanced variations in individual behaviors. While comprehensively examining group behavior, this approach offers researchers a fresh lens to discern individual behavioral traits, crucial for delving into and elucidating the intricacies of group behavior comprehensively. To validate the efficacy of this novel approach, we meticulously conducted experiments using the volleyball dataset and devised a corresponding system framework. The experimental outcomes yielded satisfactory results, showcasing the potential of our model in analyzing both group behavior and individual disparities, thus paving a novel pathway for practical applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 92%

Exploration of group activity recognition methods and system implementation

Ye,

Feng,

Wang

et al. 2024

International Conference on Computer Graphics, Artificial Intelligence, and Data Processing (ICCAID 2023)

View full text Add to dashboard Cite

show abstract

“…Recent deep learning-based GAR methods [8,[10][11][12][16][17][18][19][20] recognize the activity of a group in three steps. First, they encode the features of the states of each individual action in each frame in a video of a specific activity to obtain a feature map [21].…”

Section: Group Activity Recognitionmentioning

confidence: 99%

“…In the test set, we denote the number of samples in the k-th class as p k , k = 1, 2, • • • , K, where K is the number of classes, and the number of correctly recognized samples in the k-th class is q k . The calculations of MCA and MPCA are formulated as Equations ( 18) and (19):…”

Section: Experiments Settings 411 Datasetsmentioning

confidence: 99%

Global Individual Interaction Network Based on Consistency for Group Activity Recognition

Huang,

Zhang,

et al. 2023

Electronics

View full text Add to dashboard Cite

Modeling the interactions among individuals in a group is essential for group activity recognition (GAR). Various graph neural networks (GNNs) are regarded as popular modeling methods for GAR, as they can characterize the interaction among individuals at a low computational cost. The performance of the current GNN-based modeling methods is affected by two factors. Firstly, their local receptive field in the mapping layer limits their ability to characterize the global interactions among individuals in spatial–temporal dimensions. Secondly, GNN-based GAR methods do not have an efficient mechanism to use global activity consistency and individual action consistency. In this paper, we argue that the global interactions among individuals, as well as the constraints of global activity and individual action consistencies, are critical to group activity recognition. We propose new convolutional operations to capture the interactions among individuals from a global perspective. We use contrastive learning to maximize the global activity consistency and individual action consistency for more efficient recognition. Comprehensive experiments show that our method achieved better GAR performance than the state-of-the-art methods on two popular GAR benchmark datasets.

show abstract

“…These include spatially-detailed information, such as the person poses and object locations, and object categorical information. Such information can be readily and reliably extracted by modern deep learning algorithms and have been reported to enhance the accuracy of action recognition [6]. For example, [7] exploited the positional relations between instances and object categories, and achieved accurate scene-level object-centered action recognition.…”

Section: Introductionmentioning

confidence: 99%

Heterogeneous Feature Fusion for Improving Performance of Action Detection

Babazaki,

Iwamoto,

Takahashi

et al. 2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

We present a novel framework aimed at improving video action detection through the integration of heterogeneous features. Conventional action detection methods which focus on modeling the relationships between person/object instances rely exclusively on video features and do not exploit valuable intra-instance heterogeneous features, such as person pose, positional information or object category, that can support action recognition. Our proposed framework, termed Heterogeneous Feature Fusion (HFF) framework, addresses this limitation by integrating such intra-instance heterogeneous features for person/object instances, and can improve existing action detection methods. To efficiently exploit each heterogeneous feature, which vary in importance depending on actions and/or scenes, we introduce an attention mechanism to dynamically enhance important heterogeneous features within an instance. Experiments on JHMDB and AVA v2.2 datasets show that our HFF significantly enhances the action detection performance of two existing methods.

show abstract

COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality

Cited by 20 publications

References 39 publications

Exploration of group activity recognition methods and system implementation

Exploration of group activity recognition methods and system implementation

Global Individual Interaction Network Based on Consistency for Group Activity Recognition

Heterogeneous Feature Fusion for Improving Performance of Action Detection

Contact Info

Product

Resources

About